Luma-mapping-based video or image coding

ABSTRACT

According to the disclosure of the present document, LMCS may also be applied to a block having a dual tree structure. In addition, the number of LMCS APS may be limited. Therefore, an LMCS procedure may be efficiently performed, and the complexity of LMCS may be reduced. As the performance of LMCS is improved, the video/image coding efficiency may be increased.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of International Application NoPCT/KR2020/008047, filed Jun. 22, 2020, which claims the benefit of U.S.Patent Application No. 62/864,481, filed Jun. 20, 2019, the contents ofwhich are all hereby incorporated by reference herein in their entirety.

BACKGROUND OF DISCLOSURE Field of the Disclosure

The technique of the present document is related to luma-mapping-basedvideo or image coding.

Related Art

Recently, demand for high-resolution, high-quality image/video such as4K or 8K or higher ultra high definition (UHD) image/video has increasedin various fields. As image/video data has high resolution and highquality, the amount of information or bits to be transmitted increasesrelative to the existing image/video data, and thus, transmitting imagedata using a medium such as an existing wired/wireless broadband line oran existing storage medium or storing image/video data using existingstorage medium increase transmission cost and storage cost.

In addition, interest and demand for immersive media such as virtualreality (VR) and artificial reality (AR) content or holograms hasrecently increased and broadcasting for image/video is havingcharacteristics different from reality images such as game images hasincreased.

Accordingly, a highly efficient image/video compression technology isrequired to effectively compress, transmit, store, and reproduceinformation of a high-resolution, high-quality image/video havingvarious characteristics as described above.

In addition, a luma mapping with chroma scaling (LMCS) process isperformed to improve compression efficiency and to increasesubjective/objective visual quality, and there is a discussion forreducing computational complexity in the LMCS process.

SUMMARY

According to an embodiment of the present document, a method and anapparatus for increasing image coding efficiency are provided.

According to an embodiment of the present document, efficient filteringapplication method and apparatus are provided.

According to an embodiment of the present document, efficient LMCSapplication method and apparatus are provided.

According to an embodiment of the present document, the LMCS codewords(or a range thereof) may be constrained.

According to an embodiment of the present document, a single chromaresidual scaling factor directly signaled in chroma scaling of LMCS maybe used.

According to an embodiment of the present document, linear mapping(linear LMCS) may be used.

According to an embodiment of the present document, information on pivotpoints required for linear mapping may be explicitly signaled.

According to an embodiment of the present document, a flexible number ofbins may be used for luma mapping.

According to an embodiment of the present document, an index derivationprocedure for inverse luma mapping and/or chroma residual scaling may besimplified.

According to an embodiment of the present document, the LMCS proceduremay be applied even when luma and chroma blocks in one coding tree unit(CTU) have a separate block tree structure (dual tree structure).

According to an embodiment of the present document, the number of LMCSAPSs may be limited.

According to an embodiment of the present document, a video/imagedecoding method performed by a decoding apparatus is provided.

According to an embodiment of the present document, a decoding apparatusfor performing video/image decoding is provided.

According to an embodiment of the present document, a video/imageencoding method performed by an encoding apparatus is provided.

According to an embodiment of the present document, an encodingapparatus for performing video/image encoding is provided.

According to one embodiment of the present document, there is provided acomputer-readable digital storage medium in which encoded video/imageinformation, generated according to the video/image encoding methoddisclosed in at least one of the embodiments of the present document, isstored.

According to an embodiment of the present document, there is provided acomputer-readable digital storage medium in which encoded information orencoded video/image information, causing to perform the video/imagedecoding method disclosed in at least one of the embodiments of thepresent document by the decoding apparatus, is stored.

Advantageous Effects

According to an embodiment of the present document, overall image/videocompression efficiency may be improved.

According to an embodiment of the present document, subjective/objectivevisual quality may be improved through efficient filtering.

According to an embodiment of the present document, the LMCS process forimage/video coding may be efficiently performed.

According to an embodiment of the present document, it is possible tominimize resources/costs (of software or hardware) required for the LMCSprocess.

According to an embodiment of the present document, hardwareimplementation for the LMCS process may be facilitated.

According to an embodiment of the present document, a division operationrequired for derivation of LMCS codewords in mapping (reshaping) can beremoved or minimized by constraint of the LMCS codewords (or rangethereof).

According to an embodiment of the present document, latency according topiecewise index identification may be removed by using a single chromaresidual scaling factor.

According to an embodiment of the present document, a chroma residualscaling process can be performed without depending on (reconstructionof) a luma block by using the linear mapping in LMCS, and thus latencyin scaling can be removed.

According to an embodiment of the present document, mapping efficiencyin LMCS may be increased.

According to an embodiment of the present document, the complexity ofthe LMCS may be reduced through simplification of an index derivationprocedure for inverse luma mapping and/or chroma residual scaling, andthus video/image coding efficiency may be increased.

According to an embodiment of the present document, the LMCS proceduremay be performed even on a block having a dual tree structure, and thusthe efficiency of the LMCS may be increased. Further, coding performance(e.g., objective/subjective image quality) for a block having a dualtree structure may be improved.

According to an embodiment of the present document, as the number ofLMCS APSs is limited, the complexity of the LMCS may be reduced, andfewer resources (e.g., memory) may be consumed (used) for the LMCS.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates an example of a video/image coding system to whichthe embodiments of the present document may be applied.

FIG. 2 is a diagram schematically illustrating a configuration of avideo/image encoding apparatus to which the embodiments of the presentdocument may be applied.

FIG. 3 is a diagram schematically illustrating a configuration of avideo/image decoding apparatus to which the embodiments of the presentdocument may be applied.

FIG. 4 exemplarily shows a hierarchical structure for a codedimage/video.

FIG. 5 exemplarily illustrates a hierarchical structure of a CVSaccording to an embodiment of the present document.

FIG. 6 exemplarily illustrates a hierarchical structure of a CVSaccording to an embodiment of the present document.

FIG. 7 exemplarily illustrates a hierarchical structure of a CVSaccording to another embodiment of the present document.

FIG. 8 illustrates an exemplary LMCS structure according to anembodiment of the present document.

FIG. 9 illustrates an LMCS structure according to another embodiment ofthe present document.

FIG. 10 shows a graph representing an exemplary forward mapping.

FIG. 11 is a flowchart illustrating a method for deriving the chromaresidual scaling index according to an embodiment of the presentdocument.

FIG. 12 illustrates a linear fitting of pivot points according to anembodiment of the present document.

FIG. 13 illustrates one example of linear reshaping (or linearreshaping, linear mapping) according to an embodiment of the presentdocument.

FIG. 14 shows an example of linear forward mapping in an embodiment ofthe present document.

FIG. 15 shows an example of inverse forward mapping in an embodiment ofthe present document.

FIG. 16 and FIG. 17 schematically show an example of a video/imageencoding method and related components according to embodiment(s) of thepresent document.

FIG. 18 and FIG. 19 schematically show an example of an image/videodecoding method and related components according to an embodiment of thepresent document.

FIG. 20 shows an example of a content streaming system to whichembodiments disclosed in the present document may be applied.

FIG. 21 shows Table 16 that shows syntax and semantics of an exemplaryAPS according to an embodiment of this document.

DESCRIPTION OF EMBODIMENTS

The present document may be modified in various forms, and specificembodiments thereof will be described and shown in the drawings.However, the embodiments are not intended for limiting the presentdocument. The terms used in the following description are used to merelydescribe specific embodiments, but are not intended to limit the presentdocument. An expression of a singular number includes an expression ofthe plural number, so long as it is clearly read differently. The termssuch as “include” and “have” are intended to indicate that features,numbers, steps, operations, elements, components, or combinationsthereof used in the following description exist and it should be thusunderstood that the possibility of existence or addition of one or moredifferent features, numbers, steps, operations, elements, components, orcombinations thereof is not excluded.

Meanwhile, each configuration in the drawings described in the presentdocument is shown independently for the convenience of descriptionregarding different characteristic functions, and does not mean thateach configuration is implemented as separate hardware or separatesoftware. For example, two or more components among each component maybe combined to form one component, or one component may be divided intoa plurality of components. Embodiments in which each component isintegrated and/or separated are also included in the scope of thedisclosure of the present document.

Hereinafter, examples of the present embodiment will be described indetail with reference to the accompanying drawings. In addition, likereference numerals are used to indicate like elements throughout thedrawings, and the same descriptions on the like elements will beomitted.

FIG. 1 illustrates an example of a video/image coding system to whichthe embodiments of the present document may be applied.

Referring to FIG. 1 , a video/image coding system may include a firstdevice (a source device) and a second device (a reception device). Thesource device may transmit encoded video/image information or data tothe reception device through a digital storage medium or network in theform of a file or streaming.

The source device may include a video source, an encoding apparatus, anda transmitter. The receiving device may include a receiver, a decodingapparatus, and a renderer. The encoding apparatus may be called avideo/image encoding apparatus, and the decoding apparatus may be calleda video/image decoding apparatus. The transmitter may be included in theencoding apparatus. The receiver may be included in the decodingapparatus. The renderer may include a display, and the display may beconfigured as a separate device or an external component.

The video source may acquire video/image through a process of capturing,synthesizing, or generating the video/image. The video source mayinclude a video/image capture device and/or a video/image generatingdevice. The video/image capture device may include, for example, one ormore cameras, video/image archives including previously capturedvideo/images, and the like. The video/image generating device mayinclude, for example, computers, tablets and smartphones, and may(electronically) generate video/images. For example, a virtualvideo/image may be generated through a computer or the like. In thiscase, the video/image capturing process may be replaced by a process ofgenerating related data.

The encoding apparatus may encode input video/image. The encodingapparatus may perform a series of procedures such as prediction,transform, and quantization for compaction and coding efficiency. Theencoded data (encoded video/image information) may be output in the formof a bitstream.

The transmitter may transmit the encoded image/image information or dataoutput in the form of a bitstream to the receiver of the receivingdevice through a digital storage medium or a network in the form of afile or streaming. The digital storage medium may include variousstorage mediums such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, and thelike. The transmitter may include an element for generating a media filethrough a predetermined file format and may include an element fortransmission through a broadcast/communication network. The receiver mayreceive/extract the bitstream and transmit the received bitstream to thedecoding apparatus.

The decoding apparatus may decode the video/image by performing a seriesof procedures such as dequantization, inverse transform, and predictioncorresponding to the operation of the encoding apparatus.

The renderer may render the decoded video/image. The renderedvideo/image may be displayed through the display.

The present document relates to video/image coding. For example, amethod/embodiment disclosed in the present document may be applied to amethod disclosed in the versatile video coding (VVC) standard, theessential video coding (EVC) standard, the AOMedia Video 1 (AV 1)standard, the 2nd generation of audio video coding standard (AVS2) orthe next generation video/image coding standard (e.g., H.267, H.268, orthe like).

The present document suggests various embodiments of video/image coding,and the above embodiments may also be performed in combination with eachother unless otherwise specified.

In the present document, a video may refer to a series of images overtime. A picture generally refers to the unit representing one image at aparticular time frame, and a slice/tile refers to the unit constitutinga part of the picture in terms of coding. A slice/tile may include oneor more coding tree units (CTUs). One picture may consist of one or moreslices/tiles. One picture may consist of one or more tile groups. Onetile group may include one or more tiles. A brick may represent arectangular region of CTU rows within a tile in a picture. A tile may bepartitioned into a multiple bricks, each of which may be constructedwith one or more CTU rows within the tile. A tile that is notpartitioned into multiple bricks may also be referred to as a brick. Abrick scan may represent a specific sequential ordering of CTUspartitioning a picture, wherein the CTUs may be ordered in a CTU rasterscan within a brick, and bricks within a tile may be orderedconsecutively in a raster scan of the bricks of the tile, and tiles in apicture may be ordered consecutively in a raster scan of the tiles ofthe picture. A tile is a rectangular region of CTUs within a particulartile column and a particular tile row in a picture. The tile column is arectangular region of CTUs having a height equal to the height of thepicture and a width specified by syntax elements in the pictureparameter set. The tile row is a rectangular region of CTUs having aheight specified by syntax elements in the picture parameter set and awidth equal to the width of the picture. A tile scan is a specificsequential ordering of CTUs partitioning a picture in which the CTUs areordered consecutively in CTU raster scan in a tile whereas tiles in apicture are ordered consecutively in a raster scan of the tiles of thepicture. A slice includes an integer number of bricks of a picture thatmay be exclusively contained in a single NAL unit. A slice may consistsof either a number of complete tiles or only a consecutive sequence ofcomplete bricks of one tile. In the present document, a tile group and aslice may be used in place of each other. For example, in the presentdocument, a tile group/tile group header may be referred to as aslice/slice header.

Meanwhile, one picture may be divided into two or more subpictures. Asubpicture may be a rectangular region of one or more slices within apicture.

A pixel or a pel may mean a smallest unit constituting one picture (orimage). Also, ‘sample’ may be used as a term corresponding to a pixel. Asample may generally represent a pixel or a value of a pixel, and mayrepresent only a pixel/pixel value of a luma component or only apixel/pixel value of a chroma component.

A unit may represent a basic unit of image processing. The unit mayinclude at least one of a specific region of the picture and informationrelated to the region. One unit may include one luma block and twochroma (ex. cb, cr) blocks. The unit may be used interchangeably withterms such as block or area in some cases. In a general case, an M×Nblock may include samples (or sample arrays) or a set (or array) oftransform coefficients of M columns and N rows. Alternatively, thesample may mean a pixel value in the spatial domain, and when such apixel value is transformed to the frequency domain, it may mean atransform coefficient in the frequency domain.

In the present document, “A or B” may mean “only A”, “only B” or “both Aand B”. In other words, “A or B” in the present document may beinterpreted as “A and/or B”. For example, in the present document “A, Bor C (A, B or C)” means “only A”, “only B”, “only C”, or “anycombination of A, B and C”.

A slash (/) or comma (comma) used in the present document may mean“and/or”. For example, “A/B” may mean “A and/or B”. Accordingly, “A/B”may mean “only A”, “only B”, or “both A and B”. For example, “A, B, C”may mean “A, B, or C”.

In the present document, “at least one of A and B” may mean “only A”,“only B” or “both A and B”. Also, in the present document, theexpression “at least one of A or B” or “at least one of A and/or B” maybe interpreted the same as “at least one of A and B”.

Also, in the present document, “at least one of A, B and C” means “onlyA”, “only B”, “only C”, or “any combination of A, B and C”. Also, “atleast one of A, B or C” or “at least one of A, B and/or C” may mean “atleast one of A, B and C”.

Also, parentheses used in the present document may mean “for example”.Specifically, when “prediction (intra prediction)” is indicated, “intraprediction” may be proposed as an example of “prediction”. In otherwords, “prediction” in the present document is not limited to “intraprediction”, and “intra prediction” may be proposed as an example of“prediction”. Also, even when “prediction (i.e., intra prediction)” isindicated, “intra prediction” may be proposed as an example of“prediction”.

Technical features that are individually described in one drawing in thepresent document may be implemented individually or simultaneously.

FIG. 2 is a diagram schematically illustrating a configuration of avideo/image encoding apparatus to which the embodiments of the presentdocument may be applied. Hereinafter, what is referred to as the videoencoding apparatus may include an image encoding apparatus.

Referring to FIG. 2 , the encoding apparatus 200 includes an imagepartitioner 210, a predictor 220, a residual processor 230, and anentropy encoder 240, an adder 250, a filter 260, and a memory 270. Thepredictor 220 may include an inter predictor 221 and an intra predictor222. The residual processor 230 may include a transformer 232, aquantizer 233, a dequantizer 234, and an inverse transformer 235. Theresidual processor 230 may further include a subtractor 231. The adder250 may be called a reconstructor or a reconstructed block generator.The image partitioner 210, the predictor 220, the residual processor230, the entropy encoder 240, the adder 250, and the filter 260 may beconfigured by at least one hardware component (ex. An encoder chipset orprocessor) according to an embodiment. In addition, the memory 270 mayinclude a decoded picture buffer (DPB) or may be configured by a digitalstorage medium. The hardware component may further include the memory270 as an internal/external component.

The image partitioner 210 may partition an input image (or a picture ora frame) input to the encoding apparatus 200 into one or moreprocessors. For example, the processor may be called a coding unit (CU).In this case, the coding unit may be recursively partitioned accordingto a quad-tree binary-tree ternary-tree (QTBTTT) structure from a codingtree unit (CTU) or a largest coding unit (LCU). For example, one codingunit may be partitioned into a plurality of coding units of a deeperdepth based on a quad tree structure, a binary tree structure, and/or aternary structure. In this case, for example, the quad tree structuremay be applied first and the binary tree structure and/or ternarystructure may be applied later. Alternatively, the binary tree structuremay be applied first. The coding procedure according to the presentdisclosure may be performed based on the final coding unit that is nolonger partitioned. In this case, the largest coding unit may be used asthe final coding unit based on coding efficiency according to imagecharacteristics, or if necessary, the coding unit may be recursivelypartitioned into coding units of deeper depth and a coding unit havingan optimal size may be used as the final coding unit. Here, the codingprocedure may include a procedure of prediction, transform, andreconstruction, which will be described later. As another example, theprocessor may further include a prediction unit (PU) or a transform unit(TU). In this case, the prediction unit and the transform unit may besplit or partitioned from the aforementioned final coding unit. Theprediction unit may be a unit of sample prediction, and the transformunit may be a unit for deriving a transform coefficient and/or a unitfor deriving a residual signal from the transform coefficient.

The unit may be used interchangeably with terms such as block or area insome cases. In a general case, an M×N block may represent a set ofsamples or transform coefficients composed of M columns and N rows. Asample may generally represent a pixel or a value of a pixel, mayrepresent only a pixel/pixel value of a luma component or represent onlya pixel/pixel value of a chroma component. A sample may be used as aterm corresponding to one picture (or image) for a pixel or a pel.

In the encoding apparatus 200, a prediction signal (predicted block,prediction sample array) output from the inter predictor 221 or theintra predictor 222 is subtracted from an input image signal (originalblock, original sample array) to generate a residual signal residualblock, residual sample array), and the generated residual signal istransmitted to the transformer 232. In this case, as shown, a unit forsubtracting a prediction signal (predicted block, prediction samplearray) from the input image signal (original block, original samplearray) in the encoder 200 may be called a subtractor 231. The predictormay perform prediction on a block to be processed (hereinafter, referredto as a current block) and generate a predicted block includingprediction samples for the current block. The predictor may determinewhether intra prediction or inter prediction is applied on a currentblock or CU basis. As described later in the description of eachprediction mode, the predictor may generate various information relatedto prediction, such as prediction mode information, and transmit thegenerated information to the entropy encoder 240. The information on theprediction may be encoded in the entropy encoder 240 and output in theform of a bitstream.

The intra predictor 222 may predict the current block by referring tothe samples in the current picture. The referred samples may be locatedin the neighborhood of the current block or may be located apartaccording to the prediction mode. In the intra prediction, predictionmodes may include a plurality of non-directional modes and a pluralityof directional modes. The non-directional mode may include, for example,a DC mode and a planar mode. The directional mode may include, forexample, 33 directional prediction modes or 65 directional predictionmodes according to the degree of detail of the prediction direction.However, this is merely an example, more or less directional predictionmodes may be used depending on a setting. The intra predictor 222 maydetermine the prediction mode applied to the current block by using aprediction mode applied to a neighboring block.

The inter predictor 221 may derive a predicted block for the currentblock based on a reference block (reference sample array) specified by amotion vector on a reference picture. Here, in order to reduce theamount of motion information transmitted in the inter prediction mode,the motion information may be predicted in units of blocks, sub-blocks,or samples based on correlation of motion information between theneighboring block and the current block. The motion information mayinclude a motion vector and a reference picture index. The motioninformation may further include inter prediction direction (L0prediction, L1 prediction, Bi prediction, etc.) information. In the caseof inter prediction, the neighboring block may include a spatialneighboring block present in the current picture and a temporalneighboring block present in the reference picture. The referencepicture including the reference block and the reference pictureincluding the temporal neighboring block may be the same or different.The temporal neighboring block may be called a collocated referenceblock, a co-located CU (colCU), and the like, and the reference pictureincluding the temporal neighboring block may be called a collocatedpicture (colPic). For example, the inter predictor 221 may configure amotion information candidate list based on neighboring blocks andgenerate information indicating which candidate is used to derive amotion vector and/or a reference picture index of the current block.Inter prediction may be performed based on various prediction modes. Forexample, in the case of a skip mode and a merge mode, the interpredictor 221 may use motion information of the neighboring block asmotion information of the current block. In the skip mode, unlike themerge mode, the residual signal may not be transmitted. In the case ofthe motion vector prediction (MVP) mode, the motion vector of theneighboring block may be used as a motion vector predictor and themotion vector of the current block may be indicated by signaling amotion vector difference.

The predictor 220 may generate a prediction signal based on variousprediction methods described below. For example, the predictor may notonly apply intra prediction or inter prediction to predict one block butalso simultaneously apply both intra prediction and inter prediction.This may be called combined inter and intra prediction (CIIP). Inaddition, the predictor may be based on an intra block copy (IBC)prediction mode or a palette mode for prediction of a block. The IBCprediction mode or palette mode may be used for content image/videocoding of a game or the like, for example, screen content coding (SCC).The IBC basically performs prediction in the current picture but may beperformed similarly to inter prediction in that a reference block isderived in the current picture. That is, the IBC may use at least one ofthe inter prediction techniques described in the present disclosure. Thepalette mode may be considered as an example of intra coding or intraprediction. When the palette mode is applied, a sample value within apicture may be signaled based on information on the palette table andthe palette index.

The prediction signal generated by the predictor (including the interpredictor 221 and/or the intra predictor 222) may be used to generate areconstructed signal or to generate a residual signal. The transformer232 may generate transform coefficients by applying a transformtechnique to the residual signal. For example, the transform techniquemay include at least one of a discrete cosine transform (DCT), adiscrete sine transform (DST), a karhunen-loève transform (KLT), agraph-based transform (GBT), or a conditionally non-linear transform(CNT). Here, the GBT means transform obtained from a graph whenrelationship information between pixels is represented by the graph. TheCNT refers to transform generated based on a prediction signal generatedusing all previously reconstructed pixels. In addition, the transformprocess may be applied to square pixel blocks having the same size ormay be applied to blocks having a variable size rather than square.

The quantizer 233 may quantize the transform coefficients and transmitthem to the entropy encoder 240 and the entropy encoder 240 may encodethe quantized signal (information on the quantized transformcoefficients) and output a bitstream. The information on the quantizedtransform coefficients may be referred to as residual information. Thequantizer 233 may rearrange block type quantized transform coefficientsinto a one-dimensional vector form based on a coefficient scanning orderand generate information on the quantized transform coefficients basedon the quantized transform coefficients in the one-dimensional vectorform. Information on transform coefficients may be generated. Theentropy encoder 240 may perform various encoding methods such as, forexample, exponential Golomb, context-adaptive variable length coding(CAVLC), context-adaptive binary arithmetic coding (CABAC), and thelike. The entropy encoder 240 may encode information necessary forvideo/image reconstruction other than quantized transform coefficients(ex. values of syntax elements, etc.) together or separately. Encodedinformation (ex. encoded video/image information) may be transmitted orstored in units of NALs (network abstraction layer) in the form of abitstream. The video/image information may further include informationon various parameter sets such as an adaptation parameter set (APS), apicture parameter set (PPS), a sequence parameter set (SPS), or a videoparameter set (VPS). In addition, the video/image information mayfurther include general constraint information. In the presentdisclosure, information and/or syntax elements transmitted/signaled fromthe encoding apparatus to the decoding apparatus may be included invideo/picture information. The video/image information may be encodedthrough the above-described encoding procedure and included in thebitstream. The bitstream may be transmitted over a network or may bestored in a digital storage medium. The network may include abroadcasting network and/or a communication network, and the digitalstorage medium may include various storage media such as USB, SD, CD,DVD, Blu-ray, HDD, SSD, and the like. A transmitter (not shown)transmitting a signal output from the entropy encoder 240 and/or astorage unit (not shown) storing the signal may be included asinternal/external element of the encoding apparatus 200, andalternatively, the transmitter may be included in the entropy encoder240.

The quantized transform coefficients output from the quantizer 233 maybe used to generate a prediction signal. For example, the residualsignal (residual block or residual samples) may be reconstructed byapplying dequantization and inverse transform to the quantized transformcoefficients through the dequantizer 234 and the inverse transformer235. The adder 250 adds the reconstructed residual signal to theprediction signal output from the inter predictor 221 or the intrapredictor 222 to generate a reconstructed signal (reconstructed picture,reconstructed block, reconstructed sample array). If there is noresidual for the block to be processed, such as a case where the skipmode is applied, the predicted block may be used as the reconstructedblock. The adder 250 may be called a reconstructor or a reconstructedblock generator. The generated reconstructed signal may be used forintra prediction of a next block to be processed in the current pictureand may be used for inter prediction of a next picture through filteringas described below.

Meanwhile, luma mapping with chroma scaling (LMCS) may be applied duringpicture encoding and/or reconstruction.

The filter 260 may improve subjective/objective image quality byapplying filtering to the reconstructed signal. For example, the filter260 may generate a modified reconstructed picture by applying variousfiltering methods to the reconstructed picture and store the modifiedreconstructed picture in the memory 270, specifically, a DPB of thememory 270. The various filtering methods may include, for example,deblocking filtering, a sample adaptive offset, an adaptive loop filter,a bilateral filter, and the like. The filter 260 may generate variousinformation related to the filtering and transmit the generatedinformation to the entropy encoder 240 as described later in thedescription of each filtering method. The information related to thefiltering may be encoded by the entropy encoder 240 and output in theform of a bitstream.

The modified reconstructed picture transmitted to the memory 270 may beused as the reference picture in the inter predictor 221. When the interprediction is applied through the encoding apparatus, predictionmismatch between the encoding apparatus 200 and the decoding apparatus300 may be avoided and encoding efficiency may be improved.

The DPB of the memory 270 DPB may store the modified reconstructedpicture for use as a reference picture in the inter predictor 221. Thememory 270 may store the motion information of the block from which themotion information in the current picture is derived (or encoded) and/orthe motion information of the blocks in the picture that have alreadybeen reconstructed. The stored motion information may be transmitted tothe inter predictor 221 and used as the motion information of thespatial neighboring block or the motion information of the temporalneighboring block. The memory 270 may store reconstructed samples ofreconstructed blocks in the current picture and may transfer thereconstructed samples to the intra predictor 222.

FIG. 3 is a schematic diagram illustrating a configuration of avideo/image decoding apparatus to which the embodiment(s) of the presentdisclosure may be applied.

Referring to FIG. 3 , the decoding apparatus 300 may include an entropydecoder 310, a residual processor 320, a predictor 330, an adder 340, afilter 350, a memory 360. The predictor 330 may include an interpredictor 331 and an intra predictor 332. The residual processor 320 mayinclude a dequantizer 321 and an inverse transformer 321. The entropydecoder 310, the residual processor 320, the predictor 330, the adder340, and the filter 350 may be configured by a hardware component (ex. Adecoder chipset or a processor) according to an embodiment. In addition,the memory 360 may include a decoded picture buffer (DPB) or may beconfigured by a digital storage medium. The hardware component mayfurther include the memory 360 as an internal/external component.

When a bitstream including video/image information is input, thedecoding apparatus 300 may reconstruct an image corresponding to aprocess in which the video/image information is processed in theencoding apparatus of FIG. 2 . For example, the decoding apparatus 300may derive units/blocks based on block partition related informationobtained from the bitstream. The decoding apparatus 300 may performdecoding using a processor applied in the encoding apparatus. Thus, theprocessor of decoding may be a coding unit, for example, and the codingunit may be partitioned according to a quad tree structure, binary treestructure and/or ternary tree structure from the coding tree unit or thelargest coding unit. One or more transform units may be derived from thecoding unit. The reconstructed image signal decoded and output throughthe decoding apparatus 300 may be reproduced through a reproducingapparatus.

The decoding apparatus 300 may receive a signal output from the encodingapparatus of FIG. 2 in the form of a bitstream, and the received signalmay be decoded through the entropy decoder 310. For example, the entropydecoder 310 may parse the bitstream to derive information (ex.video/image information) necessary for image reconstruction (or picturereconstruction). The video/image information may further includeinformation on various parameter sets such as an adaptation parameterset (APS), a picture parameter set (PPS), a sequence parameter set(SPS), or a video parameter set (VPS). In addition, the video/imageinformation may further include general constraint information. Thedecoding apparatus may further decode picture based on the informationon the parameter set and/or the general constraint information.Signaled/received information and/or syntax elements described later inthe present disclosure may be decoded may decode the decoding procedureand obtained from the bitstream. For example, the entropy decoder 310decodes the information in the bitstream based on a coding method suchas exponential Golomb coding, CAVLC, or CABAC, and output syntaxelements required for image reconstruction and quantized values oftransform coefficients for residual. More specifically, the CABACentropy decoding method may receive a bin corresponding to each syntaxelement in the bitstream, determine a context model using a decodingtarget syntax element information, decoding information of a decodingtarget block or information of a symbol/bin decoded in a previous stage,and perform an arithmetic decoding on the bin by predicting aprobability of occurrence of a bin according to the determined contextmodel, and generate a symbol corresponding to the value of each syntaxelement. In this case, the CABAC entropy decoding method may update thecontext model by using the information of the decoded symbol/bin for acontext model of a next symbol/bin after determining the context model.The information related to the prediction among the information decodedby the entropy decoder 310 may be provided to the predictor (the interpredictor 332 and the intra predictor 331), and the residual value onwhich the entropy decoding was performed in the entropy decoder 310,that is, the quantized transform coefficients and related parameterinformation, may be input to the residual processor 320. The residualprocessor 320 may derive the residual signal (the residual block, theresidual samples, the residual sample array). In addition, informationon filtering among information decoded by the entropy decoder 310 may beprovided to the filter 350. Meanwhile, a receiver (not shown) forreceiving a signal output from the encoding apparatus may be furtherconfigured as an internal/external element of the decoding apparatus300, or the receiver may be a component of the entropy decoder 310.Meanwhile, the decoding apparatus according to the present disclosuremay be referred to as a video/image/picture decoding apparatus, and thedecoding apparatus may be classified into an information decoder(video/image/picture information decoder) and a sample decoder(video/image/picture sample decoder). The information decoder mayinclude the entropy decoder 310, and the sample decoder may include atleast one of the dequantizer 321, the inverse transformer 322, the adder340, the filter 350, the memory 360, the inter predictor 332, and theintra predictor 331.

The dequantizer 321 may dequantize the quantized transform coefficientsand output the transform coefficients. The dequantizer 321 may rearrangethe quantized transform coefficients in the form of a two-dimensionalblock form. In this case, the rearrangement may be performed based onthe coefficient scanning order performed in the encoding apparatus. Thedequantizer 321 may perform dequantization on the quantized transformcoefficients by using a quantization parameter (ex. quantization stepsize information) and obtain transform coefficients.

The inverse transformer 322 inversely transforms the transformcoefficients to obtain a residual signal (residual block, residualsample array).

The predictor may perform prediction on the current block and generate apredicted block including prediction samples for the current block. Thepredictor may determine whether intra prediction or inter prediction isapplied to the current block based on the information on the predictionoutput from the entropy decoder 310 and may determine a specificintra/inter prediction mode.

The predictor 320 may generate a prediction signal based on variousprediction methods described below. For example, the predictor may notonly apply intra prediction or inter prediction to predict one block butalso simultaneously apply intra prediction and inter prediction. Thismay be called combined inter and intra prediction (CIIP). In addition,the predictor may be based on an intra block copy (IBC) prediction modeor a palette mode for prediction of a block. The IBC prediction mode orpalette mode may be used for content image/video coding of a game or thelike, for example, screen content coding (SCC). The IBC basicallyperforms prediction in the current picture but may be performedsimilarly to inter prediction in that a reference block is derived inthe current picture. That is, the IBC may use at least one of the interprediction techniques described in the present disclosure. The palettemode may be considered as an example of intra coding or intraprediction. When the palette mode is applied, a sample value within apicture may be signaled based on information on the palette table andthe palette index.

The intra predictor 331 may predict the current block by referring tothe samples in the current picture. The referred samples may be locatedin the neighborhood of the current block or may be located apartaccording to the prediction mode. In the intra prediction, predictionmodes may include a plurality of non-directional modes and a pluralityof directional modes. The intra predictor 331 may determine theprediction mode applied to the current block by using a prediction modeapplied to a neighboring block.

The inter predictor 332 may derive a predicted block for the currentblock based on a reference block (reference sample array) specified by amotion vector on a reference picture. In this case, in order to reducethe amount of motion information transmitted in the inter predictionmode, motion information may be predicted in units of blocks,sub-blocks, or samples based on correlation of motion informationbetween the neighboring block and the current block. The motioninformation may include a motion vector and a reference picture index.The motion information may further include inter prediction direction(L0 prediction, L1 prediction, Bi prediction, etc.) information. In thecase of inter prediction, the neighboring block may include a spatialneighboring block present in the current picture and a temporalneighboring block present in the reference picture. For example, theinter predictor 332 may configure a motion information candidate listbased on neighboring blocks and derive a motion vector of the currentblock and/or a reference picture index based on the received candidateselection information. Inter prediction may be performed based onvarious prediction modes, and the information on the prediction mayinclude information indicating a mode of inter prediction for thecurrent block.

The adder 340 may generate a reconstructed signal (reconstructedpicture, reconstructed block, reconstructed sample array) by adding theobtained residual signal to the prediction signal (predicted block,predicted sample array) output from the predictor (including the interpredictor 332 and/or the intra predictor 331). If there is no residualfor the block to be processed, such as when the skip mode is applied,the predicted block may be used as the reconstructed block.

The adder 340 may be called reconstructor or a reconstructed blockgenerator. The generated reconstructed signal may be used for intraprediction of a next block to be processed in the current picture, maybe output through filtering as described below, or may be used for interprediction of a next picture.

Meanwhile, luma mapping with chroma scaling (LMCS) may be applied in thepicture decoding process.

The filter 350 may improve subjective/objective image quality byapplying filtering to the reconstructed signal. For example, the filter350 may generate a modified reconstructed picture by applying variousfiltering methods to the reconstructed picture and store the modifiedreconstructed picture in the memory 360, specifically, a DPB of thememory 360. The various filtering methods may include, for example,deblocking filtering, a sample adaptive offset, an adaptive loop filter,a bilateral filter, and the like.

The (modified) reconstructed picture stored in the DPB of the memory 360may be used as a reference picture in the inter predictor 332. Thememory 360 may store the motion information of the block from which themotion information in the current picture is derived (or decoded) and/orthe motion information of the blocks in the picture that have alreadybeen reconstructed. The stored motion information may be transmitted tothe inter predictor 260 so as to be utilized as the motion informationof the spatial neighboring block or the motion information of thetemporal neighboring block. The memory 360 may store reconstructedsamples of reconstructed blocks in the current picture and transfer thereconstructed samples to the intra predictor 331.

In the present document, the embodiments described in the filter 260,the inter predictor 221, and the intra predictor 222 of the encodingapparatus 200 may be the same as or respectively applied to correspondto the filter 350, the inter predictor 332, and the intra predictor 331of the decoding apparatus 300. The same may also apply to the unit 332and the intra predictor 331.

As described above, in video coding, prediction is performed to increasecompression efficiency. Through this, it is possible to generate apredicted block including prediction samples for a current block, whichis a block to be coded. Here, the predicted block includes predictionsamples in a spatial domain (or pixel domain). The predicted block isderived equally from the encoding device and the decoding device, andthe encoding device decodes information (residual information) on theresidual between the original block and the predicted block, not theoriginal sample value of the original block itself. By signaling to thedevice, image coding efficiency can be increased. The decoding apparatusmay derive a residual block including residual samples based on theresidual information, and generate a reconstructed block includingreconstructed samples by summing the residual block and the predictedblock, and generate a reconstructed picture including reconstructedblocks.

The residual information may be generated through transformation andquantization processes. For example, the encoding apparatus may derive aresidual block between the original block and the predicted block, andperform a transform process on residual samples (residual sample array)included in the residual block to derive transform coefficients, andthen, by performing a quantization process on the transformcoefficients, derive quantized transform coefficients to signal theresidual related information to the decoding apparatus (via abitstream). Here, the residual information may include locationinformation, a transform technique, a transform kernel, and aquantization parameter, value information of the quantized transformcoefficients etc. The decoding apparatus may performdequantization/inverse transformation process based on the residualinformation and derive residual samples (or residual blocks). Thedecoding apparatus may generate a reconstructed picture based on thepredicted block and the residual block. The encoding apparatus may alsodequantize/inverse transform the quantized transform coefficients forreference for inter prediction of a later picture to derive a residualblock, and generate a reconstructed picture based thereon.

In the present document, at least one of quantization/dequantizationand/or transform/inverse transform may be omitted. When thequantization/dequantization is omitted, the quantized transformcoefficient may be referred to as a transform coefficient. When thetransform/inverse transform is omitted, the transform coefficients maybe called coefficients or residual coefficients, or may still be calledtransform coefficients for uniformity of expression.

In the present document, a quantized transform coefficient and atransform coefficient may be referred to as a transform coefficient anda scaled transform coefficient, respectively. In this case, the residualinformation may include information on transform coefficient(s), and theinformation on the transform coefficient(s) may be signaled throughresidual coding syntax. Transform coefficients may be derived based onthe residual information (or information on the transformcoefficient(s)), and scaled transform coefficients may be derivedthrough inverse transform (scaling) on the transform coefficients.Residual samples may be derived based on an inverse transform(transform) of the scaled transform coefficients. This may beapplied/expressed in other parts of the present document as well.

Intra prediction may refer to prediction that generates predictionsamples for the current block based on reference samples in a picture towhich the current block belongs (hereinafter, referred to as a currentpicture). When intra prediction is applied to the current block,neighboring reference samples to be used for intra prediction of thecurrent block may be derived. The neighboring reference samples of thecurrent block may include samples adjacent to the left boundary of thecurrent block having a size of nW×nH and a total of 2×nH samplesneighboring the bottom-left, samples adjacent to the top boundary of thecurrent block and a total of 2×nW samples neighboring the top-right, andone sample neighboring the top-left of the current block. Alternatively,the neighboring reference samples of the current block may include aplurality of upper neighboring samples and a plurality of leftneighboring samples. In addition, the neighboring reference samples ofthe current block may include a total of nH samples adjacent to theright boundary of the current block having a size of nW×nH, a total ofnW samples adjacent to the bottom boundary of the current block, and onesample neighboring (bottom-right) neighboring bottom-right of thecurrent block.

However, some of the neighboring reference samples of the current blockmay not be decoded yet or available. In this case, the decoder mayconfigure the neighboring reference samples to use for prediction bysubstituting the samples that are not available with the availablesamples. Alternatively, neighboring reference samples to be used forprediction may be configured through interpolation of the availablesamples.

When the neighboring reference samples are derived, (i) the predictionsample may be derived based on the average or interpolation ofneighboring reference samples of the current block, and (ii) theprediction sample may be derived based on the reference sample presentin a specific (prediction) direction for the prediction sample among theperiphery reference samples of the current block. The case of (i) may becalled non-directional mode or non-angular mode and the case of (ii) maybe called directional mode or angular mode.

Furthermore, the prediction sample may also be generated throughinterpolation between the second neighboring sample and the firstneighboring sample located in a direction opposite to the predictiondirection of the intra prediction mode of the current block based on theprediction sample of the current block among the neighboring referencesamples. The above case may be referred to as linear interpolation intraprediction (LIP). In addition, chroma prediction samples may begenerated based on luma samples using a linear model. This case may becalled LM mode.

In addition, a temporary prediction sample of the current block may bederived based on filtered neighboring reference samples, and at leastone reference sample derived according to the intra prediction modeamong the existing neighboring reference samples, that is, unfilteredneighboring reference samples, and the temporary prediction sample maybe weighted-summed to derive the prediction sample of the current block.The above case may be referred to as position dependent intra prediction(PDPC).

In addition, a reference sample line having the highest predictionaccuracy among the neighboring multi-reference sample lines of thecurrent block may be selected to derive the prediction sample by usingthe reference sample located in the prediction direction on thecorresponding line, and then the reference sample line used herein maybe indicated (signaled) to the decoding apparatus, thereby performingintra-prediction encoding. The above case may be referred to asmulti-reference line (MRL) intra prediction or MRL based intraprediction.

In addition, intra prediction may be performed based on the same intraprediction mode by dividing the current block into vertical orhorizontal subpartitions, and neighboring reference samples may bederived and used in the subpartition unit. That is, in this case, theintra prediction mode for the current block is equally applied to thesubpartitions, and the intra prediction performance may be improved insome cases by deriving and using the neighboring reference samples inthe subpartition unit. Such a prediction method may be called intrasub-partitions (ISP) or ISP based intra prediction.

The above-described intra prediction methods may be called an intraprediction type separately from the intra prediction mode. The intraprediction type may be called in various terms such as an intraprediction technique or an additional intra prediction mode. Forexample, the intra prediction type (or additional intra prediction mode)may include at least one of the above-described LIP, PDPC, MRL, and ISP.A general intra prediction method except for the specific intraprediction type such as LIP, PDPC, MRL, or ISP may be called a normalintra prediction type. The normal intra prediction type may be generallyapplied when the specific intra prediction type is not applied, andprediction may be performed based on the intra prediction mode describedabove. Meanwhile, post-filtering may be performed on the predictedsample derived as needed.

Specifically, the intra prediction procedure may include an intraprediction mode/type determination step, a neighboring reference samplederivation step, and an intra prediction mode/type based predictionsample derivation step. In addition, a post-filtering step may beperformed on the predicted sample derived as needed.

When intra prediction is applied, the intra prediction mode applied tothe current block may be determined using the intra prediction mode ofthe neighboring block. For example, the decoding apparatus may selectone of most probable mode (mpm) candidates of an mpm list derived basedon the intra prediction mode of the neighboring block (ex. left and/orupper neighboring blocks) of the current block based on the received mpmindex and select one of the other remaining intro prediction modes notincluded in the mpm candidates (and planar mode) based on the remainingintra prediction mode information. The mpm list may be configured toinclude or not include a planar mode as a candidate. For example, if thempm list includes the planar mode as a candidate, the mpm list may havesix candidates. If the mpm list does not include the planar mode as acandidate, the mpm list may have three candidates. When the mpm listdoes not include the planar mode as a candidate, a not planar flag (ex.intra_luma_not_planar_flag) indicating whether an intra prediction modeof the current block is not the planar mode may be signaled. Forexample, the mpm flag may be signaled first, and the mpm index and notplanar flag may be signaled when the value of the mpm flag is 1. Inaddition, the mpm index may be signaled when the value of the not planarflag is 1. Here, the mpm list is configured not to include the planarmode as a candidate does not is to signal the not planar flag first tocheck whether it is the planar mode first because the planar mode isalways considered as mpm.

For example, whether the intra prediction mode applied to the currentblock is in mpm candidates (and planar mode) or in remaining mode may beindicated based on the mpm flag (ex. Intra_luma_mpm_flag). A value 1 ofthe mpm flag may indicate that the intra prediction mode for the currentblock is within mpm candidates (and planar mode), and a value 0 of thempm flag may indicate that the intra prediction mode for the currentblock is not in the mpm candidates (and planar mode). The value 0 of thenot planar flag (ex. Intra_luma_not_planar_flag) may indicate that theintra prediction mode for the current block is planar mode, and thevalue 1 of the not planar flag value may indicate that the intraprediction mode for the current block is not the planar mode. The mpmindex may be signaled in the form of an mpm_idx or intra_luma_mpm_idxsyntax element, and the remaining intra prediction mode information maybe signaled in the form of a rem_intra_luma_pred_mode orintra_luma_mpm_remainder syntax element. For example, the remainingintra prediction mode information may index remaining intra predictionmodes not included in the mpm candidates (and planar mode) among allintra prediction modes in order of prediction mode number to indicateone of them. The intra prediction mode may be an intra prediction modefor a luma component (sample). Hereinafter, intra prediction modeinformation may include at least one of the mpm flag (ex.Intra_luma_mpm_flag), the not planar flag (ex.Intra_luma_not_planar_flag), the mpm index (ex. mpm_idx orintra_luma_mpm_idx), and the remaining intra prediction mode information(rem_intra_luma_pred_mode or intra_luma_mpm_remainder). In the presentdocument, the MPM list may be referred to in various terms such as MPMcandidate list and candModeList. When MIP is applied to the currentblock, a separate mpm flag (ex. intra_mip_mpm_flag), an mpm index (ex.intra_mip_mpm_idx), and remaining intra prediction mode information (ex.intra_mip_mpm_remainder) for MIP may be signaled and the not planar flagis not signaled.

In other words, in general, when block splitting is performed on animage, a current block and a neighboring block to be coded have similarimage characteristics. Therefore, the current block and the neighboringblock have a high probability of having the same or similar intraprediction mode. Thus, the encoder may use the intra prediction mode ofthe neighboring block to encode the intra prediction mode of the currentblock.

For example, the encoder/decoder may configure a list of most probablemodes (MPM) for the current block. The MPM list may also be referred toas an MPM candidate list. Herein, the MPM may refer to a mode used toimprove coding efficiency in consideration of similarity between thecurrent block and neighboring block in intra prediction mode coding. Asdescribed above, the MPM list may be configured to include the planarmode or may be configured to exclude the planar mode. For example, whenthe MPM list includes the planar mode, the number of candidates in theMPM list may be 6. And, if the MPM list does not include the planarmode, the number of candidates in the MPM list may be 5.

The encoder/decoder may configure an MPM list including 5 or 6 MPMs.

In order to configure the MPM list, three types of modes can beconsidered: default intra modes, neighbor intra modes, and the derivedintra modes.

For the neighboring intra modes, two neighboring blocks, i.e., a leftneighboring block and an upper neighboring block, may be considered.

As described above, if the MPM list is configured not to include theplanar mode, the planar mode is excluded from the list, and the numberof MPM list candidates may be set to 5.

In addition, the non-directional mode (or non-angular mode) among theintra prediction modes may include a DC mode based on the average ofneighboring reference samples of the current block or a planar modebased on interpolation.

When inter prediction is applied, the predictor of the encodingapparatus/decoding apparatus may derive a prediction sample byperforming inter prediction in units of blocks. Inter prediction may bea prediction derived in a manner that is dependent on data elements (ex.sample values or motion information) of picture(s) other than thecurrent picture. When inter prediction is applied to the current block,a predicted block (prediction sample array) for the current block may bederived based on a reference block (reference sample array) specified bya motion vector on the reference picture indicated by the referencepicture index. Here, in order to reduce the amount of motion informationtransmitted in the inter prediction mode, the motion information of thecurrent block may be predicted in units of blocks, subblocks, or samplesbased on correlation of motion information between the neighboring blockand the current block. The motion information may include a motionvector and a reference picture index. The motion information may furtherinclude inter prediction type (L0 prediction, L1 prediction, Biprediction, etc.) information. In the case of inter prediction, theneighboring block may include a spatial neighboring block present in thecurrent picture and a temporal neighboring block present in thereference picture. The reference picture including the reference blockand the reference picture including the temporal neighboring block maybe the same or different. The temporal neighboring block may be called acollocated reference block, a co-located CU (colCU), and the like, andthe reference picture including the temporal neighboring block may becalled a collocated picture (colPic). For example, a motion informationcandidate list may be configured based on neighboring blocks of thecurrent block, and flag or index information indicating which candidateis selected (used) may be signaled to derive a motion vector and/or areference picture index of the current block. Inter prediction may beperformed based on various prediction modes. For example, in the case ofa skip mode and a merge mode, the motion information of the currentblock may be the same as motion information of the neighboring block. Inthe skip mode, unlike the merge mode, the residual signal may not betransmitted. In the case of the motion vector prediction (MVP) mode, themotion vector of the selected neighboring block may be used as a motionvector predictor and the motion vector of the current block may besignaled. In this case, the motion vector of the current block may bederived using the sum of the motion vector predictor and the motionvector difference.

The motion information may include L0 motion information and/or L1motion information according to an inter prediction type (L0 prediction,L1 prediction, Bi prediction, etc.). The motion vector in the L0direction may be referred to as an L0 motion vector or MVL0, and themotion vector in the L1 direction may be referred to as an L1 motionvector or MVL1. Prediction based on the L0 motion vector may be calledL0 prediction, prediction based on the L1 motion vector may be called L1prediction, and prediction based on both the L0 motion vector and the L1motion vector may be called bi-prediction. Here, the L0 motion vectormay indicate a motion vector associated with the reference picture listL0 (L0), and the L1 motion vector may indicate a motion vectorassociated with the reference picture list L1 (L1). The referencepicture list L0 may include pictures that are earlier in output orderthan the current picture as reference pictures, and the referencepicture list L1 may include pictures that are later in the output orderthan the current picture. The previous pictures may be called forward(reference) pictures, and the subsequent pictures may be called reverse(reference) pictures. The reference picture list L0 may further includepictures that are later in the output order than the current picture asreference pictures. In this case, the previous pictures may be indexedfirst in the reference picture list L0 and the subsequent pictures maybe indexed later. The reference picture list L1 may further includeprevious pictures in the output order than the current picture asreference pictures. In this case, the subsequent pictures may be indexedfirst in the reference picture list 1 and the previous pictures may beindexed later. The output order may correspond to picture order count(POC) order.

FIG. 4 exemplarily shows a hierarchical structure for a codedimage/video.

Referring to FIG. 4 , coded image/video is divided into a video codinglayer (VCL) that handles the decoding process of the image/video anditself, a subsystem that transmits and stores the coded information, andNAL (network abstraction layer) in charge of function and presentbetween the VCL and the subsystem.

In the VCL, VCL data including compressed image data (slice data) isgenerated, or a parameter set including a picture parameter set (PSP), asequence parameter set (SPS), and a video parameter set (VPS) or asupplemental enhancement information (SEI) message additionally requiredfor an image decoding process may be generated.

In the NAL, a NAL unit may be generated by adding header information(NAL unit header) to a raw byte sequence payload (RBSP) generated in aVCL. In this case, the RBSP refers to slice data, parameter set, SEImessage, etc., generated in the VCL. The NAL unit header may include NALunit type information specified according to RBSP data included in thecorresponding NAL unit.

As shown in the figure, the NAL unit may be classified into a VCL NALunit and a Non-VCL NAL unit according to the RBSP generated in the VCL.The VCL NAL unit may mean a NAL unit that includes information on theimage (slice data) on the image, and the Non-VCL NAL unit may mean a NALunit that includes information (parameter set or SEI message) requiredfor decoding the image.

The above-described VCL NAL unit and Non-VCL NAL unit may be transmittedthrough a network by attaching header information according to the datastandard of the subsystem. For example, the NAL unit may be transformedinto a data format of a predetermined standard such as an H.266/VVC fileformat, a real-time transport protocol (RTP), a transport stream (TS),etc., and transmitted through various networks.

As described above, the NAL unit may be specified with the NAL unit typeaccording to the RBSP data structure included in the corresponding NALunit, and information on the NAL unit type may be stored and signaled inthe NAL unit header.

For example, the NAL unit may be classified into a VCL NAL unit type anda Non-VCL NAL unit type according to whether the NAL unit includesinformation (slice data) about an image. The VCL NAL unit type may beclassified according to the nature and type of pictures included in theVCL NAL unit, and the Non-VCL NAL unit type may be classified accordingto types of parameter sets.

The following is an example of the NAL unit type specified according tothe type of parameter set included in the Non-VCL NAL unit type.

-   -   APS (Adaptation Parameter Set) NAL unit: Type for NAL unit        including APS    -   DPS (Decoding Parameter Set) NAL unit: Type for NAL unit        including DPS    -   VPS (Video Parameter Set) NAL unit: Type for NAL unit including        VPS    -   SPS(Sequence Parameter Set) NAL unit: Type for NAL unit        including SPS    -   PPS (Picture Parameter Set) NAL unit: Type for NAL unit        including PPS    -   PH (Picture header) NAL unit: Type for NAL unit including PH

The aforementioned NAL unit types may have syntax information for theNAL unit type, and the syntax information may be stored and signaled ina NAL unit header. For example, the syntax information may benal_unit_type, and NAL unit types may be specified by a nal_unit_typevalue.

Meanwhile, as described above, one picture may include a plurality ofslices, and one slice may include a slice header and slice data. In thiscase, one picture header may be further added to a plurality of slices(a slice header and a slice data set) in one picture. The picture header(picture header syntax) may include information/parameters commonlyapplicable to the picture. In the present document, a slice may be mixedor replaced with a tile group. Also, in the present document, a sliceheader may be mixed or replaced with a tile group header.

The slice header (slice header syntax) may includeinformation/parameters that may be commonly applied to the slice. TheAPS (APS syntax) or the PPS (PPS syntax) may includeinformation/parameters that may be commonly applied to one or moreslices or pictures. The SPS (SPS syntax) may includeinformation/parameters that may be commonly applied to one or moresequences. The VPS (VPS syntax) may include information/parameters thatmay be commonly applied to multiple layers. The DPS (DPS syntax) mayinclude information/parameters that may be commonly applied to theoverall video. The DPS may include information/parameters related toconcatenation of a coded video sequence (CVS). The high level syntax(HLS) in the present document may include at least one of the APSsyntax, the PPS syntax, the SPS syntax, the VPS syntax, the DPS syntax,and the slice header syntax.

In the present document, the image/image information encoded from theencoding apparatus and signaled to the decoding apparatus in the form ofa bitstream includes not only partitioning related information in apicture, intra/inter prediction information, residual information,in-loop filtering information, etc, but also information included in aslice header, information included in the APS, information included inthe PPS, information included in an SPS, and/or information included inthe VPS.

Meanwhile, in order to compensate for a difference between an originalimage and a reconstructed image due to an error occurring in acompression coding process such as quantization, an in-loop filteringprocess may be performed on reconstructed samples or reconstructedpictures as described above. As described above, the in-loop filteringmay be performed by the filter of the encoding apparatus and the filterof the decoding apparatus, and a deblocking filter, SAO, and/or adaptiveloop filter (ALF) may be applied. For example, the ALF process may beperformed after the deblocking filtering process and/or the SAO processare completed. However, even in this case, the deblocking filteringprocess and/or the SAO process may be omitted.

Meanwhile, in order to increase coding efficiency, luma mapping withchroma scaling (LMCS) may be applied as described above. LMCS may bereferred to as a loop reshaper (reshaping). In order to increase codingefficiency, LMCS control and/or signaling of LMCS related informationmay be performed hierarchically.

FIG. 5 exemplarily shows a hierarchical structure of CVS according to anembodiment of this document.

Referring to FIG. 5 , a coded video sequence (CVS) may include an SPS,one or more sequence parameter sets (PPSs), and one or more subsequentcoded pictures. Each coded picture may be divided into rectangularregions. The rectangular regions may be referred to as tiles. Gatheredone or more tiles may form a tile group or slice. In this case, the tilegroup header may be linked to a picture parameter set (PPS), and the PPSmay be linked to the SPS.

FIG. 6 exemplarily illustrates a hierarchical structure of a CVSaccording to an embodiment of the present document. A coded videosequence (CVS) may include an SPS, a PPS, a tile group header, tiledata, and/or CTU(s). Here, the tile group header and the tile data maybe referred to as a slice header and slice data, respectively.

The SPS may include flags natively to enable tools to be used in CVS. Inaddition, the SPS may be referred to by the PPS including information onparameters that change for each picture. Each of the coded pictures mayinclude one or more coded rectangular domain tiles. The tiles may begrouped into raster scans forming tile groups. Each tile group isencapsulated with header information called a tile group header. Eachtile consists of a CTU comprising coded data. Here the data may includeoriginal sample values, prediction sample values, and its luma andchroma components (luma prediction sample values and chroma predictionsample values).

According to the existing method, ALF data (ALF parameter) or LMCS data(LMCS parameter) was incorporated in the tile group header. Because onevideo is constituted by multiple pictures and one picture includesmultiple tiles, signaling ALF data (ALF parameter) or LMCS data (LMCSparameter) frequently in units of tile groups has led to the problem ofthe reduction of the coding efficiency.

According to an embodiment proposed in this document, the ALF parameteror LMCS data (LMCS parameter) may be incorporated in an APS and signaledas follows.

FIG. 7 exemplarily shows a hierarchical structure of CVS according toanother embodiment of this document.

Referring to FIG. 7 , an APS is defined, and the APS may carry necessaryALF data (ALF parameter). Furthermore, the APS may haveself-identification parameters, ALF data, and/or LMCS data. Theself-identification parameter of the APS may include an APS ID. That is,the APS may include information representing the APS ID. The tile groupheader or the slice header may make reference to the APS using APS indexinformation. In other words, the tile group header or the slice headermay include APS index information, and may perform the ALF procedure forthe target block based on the LMCS data (LMCS parameter) included in theAPS having the APS ID indicated by the APS index information, or theLMCS procedure for the target block may be performed based on the ALFdata (ALF parameter) included in the APS having the APS ID indicated bythe APS index information. Here, the APS index information may bereferred to as APS ID information.

In an example, the SPS may include a flag allowing the use of the ALF.For example, when the CVS begins, the SPS may be checked, and the flagin the SPS may be checked. For example, the SPS may include the syntaxof Table 1 below. The syntax in Table 1 may be a part of the SPS.

TABLE 1 Descriptor seq_parameter_set_rbsp( ) { ...  sps_alf_enabled_flag u(1) }

The semantics of syntax elements included in the syntax of Table 1 maybe represented, for example, as shown in the following table.

TABLE 2 sps_alf_enabled_flag equal to 0 specifies that the adaptive loopfilter is disabled. sps_alf_enabled_flag equal to 1 specifies that theadaptive loop filter is enabled.

That is, the sps_alf_enabled_flag syntax element may indicate whether ornot the ALF is enabled, based on whether the value thereof is 0 or 1.The sps_alf_enabled_flag syntax element may be referred to as an ALFenabled flag (which may be referred to as a first ALF enabled flag), andmay be included in the SPS. That is, the ALF enabled flag may besignaled in the SPS (or SPS level). When the value of the ALF enabledflag signaled in the SPS is 1, the ALF may be determined to be basicallyenabled for pictures in the CVS making reference to the SPS. Meanwhile,as described above, the ALF may be separately turned on/off by signalingan additional enabled flag at a lower level than the SPS.

For example, if the ALF tool is enabled for CVS, an additional enabledflag (which may be called a second ALF enabled flag) may be signaled ina tile group header or a slice header. The second ALF enabled flag maybe parsed/signaled when, for example, the ALF is enabled at the SPSlevel. When the value of the second ALF enabled flag is 1, ALF data maybe parsed through the tile group header or the slice header. Forexample, the second ALF enabled flag may specify an ALF enabledcondition for luma and chroma components. The ALF data may be accessedthrough APS ID information.

TABLE 3 Descriptor tile_group_header( ) { tile_group_pic_parameter_set_id  ue(v)  if( sps_alf_enabled_flag ) {  tile_group_alf_enabled_flag  u(1)   if( tile_group_alf_enabled_flag )   tile_group_aps_id  u(5)  }

TABLE 4 Descriptor slice_header( ) {  slice_pic_parameter_set_id  ue(v) if( sps_alf_enabled_flag) {   slice_alf_enabled_flag  u(1)   if(slice_alf_enabled_flag )    slice_aps_id  u(5)  }

The semantics of syntax elements included in the syntax of Table 3 or 4above may be represented, for example, as shown in the following tables.

TABLE 5 tile_group_alf_enabled_flag equal to 1 specifies that adaptiveloop filter is enabled and may be applied to Y, Cb, or Cr colorcomponent in a tile group. tile_group_alf_enabled_flag equal to 0specifies that adaptive loop filter is disabled for all color componentsin a tile group. tile_group_aps_id specifies theadaptation_parameter_set_id of the APS that the tile group refers to.The TemporalId of the APS NAL unit having adaptation_parameter_set_idequal to tile_group_aps_id shall be less than or equal to the TemporalIdof the coded tile group NAL unit. When multiple APSs with the same valueof adaptation_parameter_set_id are referred to by two or more tilegroups of the same picture, the multiple APSs with the same value ofadaptation_parameter_set_id shall have the same content.

TABLE 6 slice_alf_enabled_flag equal to 1 specifies that adaptive loopfilter is enabled and may be applied to Y, Cb, or Cr color component ina slice. slice_alf_enabled_flag equal to 0 specifies that adaptive loopfilter is disabled for all color components in a slice. slice_aps_idspecifies the adaptation_parameter_set_id of the APS that the slicerefers to. The TemporalId of the APS NAL unit havingadaptation_parameter_set_id equal to tile_group_aps_id shall be lessthan or equal to the TemporalId of the coded slice NAL unit. Whenmultiple APSs with the same value of adaptation_parameter_set_id arereferred to by two or more slices of the same picture, the multiple APSswith the same value of adaptation_parameter_set_id shall have the samecontent.

The second ALF enabled flag may include a tile_group_alf_enabled_flagsyntax element or a slice_alf_enabled_flag syntax element.

An APS referenced by a corresponding tile group or a corresponding slicemay be identified based on the APS ID information (e.g., atile_group_aps_id syntax element or a slice_aps_id syntax element). TheAPS may include ALF data.

Meanwhile, the structure of the APS including the ALF data may bedescribed based on the following syntax and semantics, for example. Thesyntax of Table 7 may be a part of the APS.

TABLE 7 Descriptor adaptation_parameter_set_rbsp( ) { adaptation_parameter_set_id  u(5)  alf_data( )  aps_extension_flag u(1)  if( aps_extension_flag )   while( more_rbsp_data( ) )   aps_extension_data_flag  u(1)  rbsp_trailing_bits( ) }

TABLE 8 adaptation_parameter_set_id provides an identifier for the APSfor reference by other syntax elements. aps_extension_flag equal to 0specifies that no aps_extension_data_flag syntax elements are present inthe APS RBSP syntax structure. aps_extension_flag equal to 1 specifiesthat there are aps_extension_data_flag syntax elements present in theAPS RBSP syntax structure. aps_extension_data_flag may have any valueits presence and value do not affect decoder conformance to profilesspecified in this version of this Specification. Decoders conforming tothis version of this Specification shall ignore allaps_extension_data_flag syntax elements. NOTE APSs can be shared acrosspictures and can be different in different tile groups within a picture.

As described above, the adaptation_parameter_set_id syntax element mayrepresent the identifier of the corresponding APS. That is, the APS maybe identified based on the adaptation_parameter_set_id syntax element.The adaptation_parameter_set_id syntax element may be referred to as APSID information. Also, the APS may include an ALF data field. The ALFdata field may be parsed/signaled after the adaptation_parameter_set_idsyntax element.

Additionally, for example, an APS extension flag (e.g.,aps_extension_flag syntax element) may be parsed/signaled in the APS.The APS extension flag may indicate whether or not APS extension dataflag (aps_extension_data_flag) syntax elements are present. The APSextension flag may be used, for example, to provide extension points fora later version of the VVC standard.

FIG. 8 illustrates an exemplary LMCS structure according to anembodiment of the present document. The LMCS structure 800 of FIG. 8includes an in-loop mapping part 810 of luma components based onadaptive piecewise linear (adaptive PWL) models and a luma-dependentchroma residual scaling part 820 for chroma components. Thedequantization and inverse transform 811, reconstruction 812, and intraprediction 813 blocks of the in-loop mapping part 810 representprocesses applied in the mapped (reshaped) domain. Loop filters 815,motion compensation or inter prediction 817 blocks of the in-loopmapping part 810, and reconstruction 822, intra prediction 823, motioncompensation or inter prediction 824, loop filters 825 block of thechroma residual scaling part 820 represent processes applied in theoriginal (non-mapped, non-reshaped) domain.

As illustrated in FIG. 8 , when LMCS is enabled, at least one of theinverse mapping (reshaping) process 814, a forward mapping (reshaping)process 818, and a chroma scaling process 821 may be applied. Forexample, the inverse mapping process may be applied to a (reconstructed)luma sample (or luma samples or luma sample array) in a reconstructedpicture. The inverse mapping process may be performed based on apiecewise function (inverse) index of a luma sample. The piecewisefunction (inverse) index may identify the piece to which the luma samplebelongs. Output of the inverse mapping process is a modified(reconsturcted) luma sample (or modified luma samples or modified lumasample array). The LMCS may be enabled or disabled at a level of a tilegroup (or slice), picture or higher.

The forward mapping process and/or the chroma scaling process may beapplied to generate the reconstructed picture. A picture may compriseluma samples and chroma samples. A reconstructed picture with lumasamples may be referred to as a reconstructed luma picture, and areconstructed picture with chroma samples may be referred to as areconstructed chroma picture. A combination of the reconstructed lumapicture and the reconstructed chroma picture may be referred to as areconstructed picture. The reconstructed luma picture may be generatedbased on the forward mapping process. For example, if an interprediction is applied to a current block, a forward mapping is appliedto a luma prediction sample derived based on a (reconstructed) lumasample in a reference picture. Because the (reconstructed) luma samplein the reference picture is generated based on the inverse mappingprocess, the forward mapping may be applied to the luma predictionsample thus a mapped (reshaped) luma prediction sample can be derived.The forward mapping process may be performed based on a piecewisefunction index of the luma prediction sample. The piecewise functionindex may be derived based on the value of the luma prediction sample orthe value of the luma sample in the reference picture used for interprediction. If an intra prediction (or an intra block copy (IBC)) isapplied to the current block, the forward mapping is not necessarybecause the inverse mapping process has not applied to the reconstructedsamples in the current picture yet. A (reconstructed) luma sample in thereconstructed luma picture is generated based on the mapped lumaprediction sample and a corresponding luma residual sample.

The reconstructed chroma picture may be generated based on the chromascaling process. For example, a (reconstructed) chroma sample in thereconstructed chroma picture may be derived based on a chroma predictionsample and a chroma residual sample (c_(res)) in a current block. Thechroma residual sample (c_(res)) is derived based on a (scaled) chromaresidual sample (c_(resScale)) and a chroma residual scaling factor(cScaleInv may be referred to as varScale) for the current block. Thechroma residual scaling factor may be calculated based on reshaped lumaprediction sample values for the current block. For example, the scalingfactor may be calculated based on an average luma value ave(Y′_(pred))of the reshaped luma prediction sample values Y′_(pred). For areference, the (scaled) chroma residual sample derived based on theinverse transform/dequantization may be referred to as c_(resScale), andthe chroma residual sample derived by performing the (inverse) scalingprocess to the (scaled) chroma residual sample may be referred to asc_(res).

FIG. 9 illustrates an LMCS structure according to another embodiment ofthe present document. FIG. 9 is described with reference to FIG. 8 .Here, the difference between the LMCS structure of FIG. 9 and the LMCSstructure 800 of FIG. 8 is mainly described. The in-loop mapping partand the luma-dependent chroma residual scaling part of FIG. 9 mayoperate the same as (similarly to) the in-loop mapping part 810 and theluma-dependent chroma residual scaling part 820 of FIG. 8 .

Referring to FIG. 9 , a chroma residual scaling factor may be derivedbased on luma reconstructed samples. In this case, an average luma value(avgYr) may be obtained (derived) based on the neighboring lumareconstructed samples outside the reconstructed block, not the innerluma reconstructed samples of the reconstructed block, and the chromaresidual scaling factor is derived based on the average luma value(avgYr). Here, the neighboring luma reconstructed samples may beneighboring luma reconstructed samples of the current block, or may beneighboring luma reconstructed samples of virtual pipeline data units(VPDUs) including the current block. For example, when intra predictionis applied to the target block, reconstructed samples may be derivedbased on prediction samples which are derived based on the intraprediction. In the other example, when inter prediction is applied tothe target block, the forward mapping is applied to prediction sampleswhich are derived based on the inter prediction, and reconstructedsamples are generated (derived) based on the reshaped (or forwardmapped) luma prediction samples.

The video/image information signaled through the bitstream may includeLMCS parameters (information on LMCS). LMCS parameters may be configuredas high level syntax (HLS, including slice header syntax) or the like.Detailed description and configuration of the LMCS parameters will bedescribed later. As described above, the syntax tables described in thepresent document (and the following embodiments) may beconfigured/encoded at the encoder end and signaled to the decoder endthrough a bitstream. The decoder may parse/decode information on theLMCS (in the form of syntax components) in the syntax tables. One ormore embodiments to be described below may be combined. The encoder mayencode the current picture based on the information about the LMCS andthe decoder may decode the current picture based on the informationabout the LMCS.

The in-loop mapping of luma components may adjust the dynamic range ofthe input signal by redistributing the codewords across the dynamicrange to improve compression efficiency. For luma mapping, a forwardmapping (reshaping) function (FwdMap) and an inverse mapping (reshaping)function (InvMap) corresponding to the forward mapping function (FwdMap)may be used. The FwdMap function may be signaled using a piece-wiselinear models, for example, the piece-wise linear model may have 16pieces or bins. The pieces may have the equal length. In one example,the InvMap function does not need to be signalled and is instead derivedfrom the FwdMap function. That is, the inverse mapping may be a functionof the forward mapping. For example, the inverse mapping function may bemathematically built as the symmetric function of the forward mapping asreflected by the line y=x.

An in-loop (luma) reshaping may be used to map input luma values(samples) to altered values in the reshaped domain. The reshaped valuesmay be coded and then mapped back into the original (un-mapped,un-reshaped) domain after reconstruction. To compensate for theinteraction between the luma signal and the chroma signal, chromaresidual scaling may be applied. In-loop reshaping is done by specifyinghigh level syntax for the reshaper model. The reshaper model syntax maysignal a piece-wise linear model (PWL model). For example, the reshapermodel syntax may signal a PWL model with 16 bins or pieces of equallengths. A forward lookup table (FwdLUT) and/or an inverse lookup table(InvLUT) may be derived based on the piece-wise linear model. Forexample, the PWL model pre-computes the 1024-entry forward (FwdLUT) andinverse (InvLUT) look up tables (LUT)s. As an example, when the forwardlookup table FwdLUT is derived, the inverse lookup table InvLUT may bederived based on the forward lookup table FwdLUT. The forward lookuptable FwdLUT may map the input luma values Yi to the altered values Yr,and the inverse lookup table InvLUT may map the altered values Yr to thereconstructed values Y′i. The reconstructed values Y′i may be derivedbased on the input luma values Yi.

In one example, the SPS may include the syntax of Table 9 below. Thesyntax of Table may include sps_reshaper_enabled_flag as a tool enablingflag. Here, sps_reshaper_enabled_flag may be used to specify whether thereshaper is used in a coded video sequence (CVS). That is,sps_reshaper_enabled_flag may be a flag for enabling reshaping in theSPS. In one example, the syntax of Table 9 may be a part of the SPS.

TABLE 9 Descriptor seq_parameter_set_rbsp( ) {  sps_seq_parameter_set_id ue(v) ...  sps_reshaper_enabled_flag  u(1)  rbsp_trailing_bits( ) }

In one example, semantics on syntax elements sps_seq_parameter_set_idand sps_reshaper_enabled_flag may be as shown in Table 10 below.

TABLE 10 sps_seq_parameter_set_id provides an identifier for the SPS forreference by other syntax elements. sps_reshaper_enabled_flag equal to 1specifies that reshaper is used in the coded video sequence (CVS).sps_reshaper_enabled_flag equal to 0 specifies that reshaper is not usedin the CVS.

In one example, the tile group header or the slice header may includethe syntax of Table 11 or Table 12 below.

TABLE 11 Descriptor tile_group_header( ) { tile_group_pic_parameter_set_id ue(v) ...  if(num_tiles_in_tile_group_minus1 > 0) {   offset_len_minus1 ue(v)   for( i= 0, i < num_tiles_in_tile_group_minus1, i++ )   entry_point_offset_minus1[ i ] u(v)  }  if (sps_reshaper_enabled_flag ) {   tile_group_reshaper_model_present_flagu(1)   if ( tile_group_reshaper_model_present_flag )   tile_group_reshaper_model ( )   tile_group_reshaper_enable_flag u(1)  if ( tile_group_reshaper_enable_flag && (!( qtbtt_dual_tree_intra_flag&& tile_group_type == I ) ) )   tile_group_reshaper_chroma_residual_scale_flag u(1)  } byte_alignment( ) }

TABLE 12 Descriptor slice header( ) {  slice_pic_parameter_set_id ue(v)...  if( num_tiles_in_slice_minus1 > 0 ) {  offset_len_minus1 ue(v) for( i = 0, i > num_tiles_in_slice_minus1; i++ )  entry_point_offset_minus1[ i ] u(v)  }  if( sps_reshaper_enabled_flag){   slice_reshaper_model_present_flag u(1)   if(slice_reshaper_model_present_flag    slice_reshaper_model ( )  slice_reshaper_enable_flag u(1)   if (slice_reshaper_enable_flag &&(!( qtbtt_dual_tree_intra_flag && slice_type == I ) ) )   slice_reshaper_chroma_residual_scale_flag u(1)  }  byte_alignment( )}

Semantics of syntax elements included in the syntax of Table 11 or Table12 may include, for example, matters disclosed in the following tables.

TABLE 13 tile_group_reshaper_model_present_flag equal to 1 specifiestile_group_reshaper_model ( ) is present in tile group header.tile_group_reshaper_model_present_flag equal to 0 sp ecifiestile_group_reshaper_model( ) is not present in tile group header. Whentile_group _reshaper_model_present_flag is not present, it is inferredto be equal to 0. tile_group_reshaper_enabled_flag equal to 1 specifiesthat reshaper is enabled for the current tile group.tile_group_reshaper_enabled_flag equal to 0 specifies that reshaper is not enabled for the current tile group. Whentile_group_reshaper_enable_flag is not pr esent, it is inferred to beequal to 0. tile_group_reshaper_chroma_residual_scale_flag equal to 1specifies that chroma resid ual scaling is enabled for the current tilegroup. tile_group_reshaper_chroma_residual_s cale_flag equal to 0specifies that chroma residual scaling is not enabled for the curre nttile group. When tile_group_reshaper_chroma_residual_scale_flag is notpresent, it is inferred to be equal to 0.

TABLE 14 slice_reshaper_model_present_flag equal to 1 specifiesslice_reshaper_model( ) is present in slice header.slice_reshaper_model_present_flag equal to 0 specifiesslice_reshaper_model( ) is not present in slice header. Whenslice_reshaper_model_present_flag is not present, it is inferred to beequal to 0. slice_reshaper_enabled_flag equal to 1 specifies thatreshaper is enabled for the current slice. slice_reshaper_enabled_flagequal to 0 specifies that reshaper is not enabled for the current slice.When slice_reshaper_enable_flag is not present, it is inferred to beequal to 0. slice_reshaper_chroma_residual_scale_flag equal to 1specifies that chroma residual scaling is enabled for the current slice.slice_reshaper_chroma_residual_scale_flag equal to 0 specifies thatchroma residual scaling is not enabled for the current slice. Whenslice_reshaper_chroma_residual_scale_flag is not present it is inferredto be equal to 0.

As one example, once the flag enabling the reshaping (i.e.,sps_reshaper_enabled_flag) is parsed in the SPS, the tile group headermay parse additional data (i.e., information included in Table 13 or 14above) which is used to construct lookup tables (FwdLUT and/or InvLUT).In order to do this, the status of the SPS reshaper flag(sps_reshaper_enabled_flag) may be first checked in the slice header orthe tile group header. When sps_reshaper_enabled_flag is true (or 1), anadditional flag, i.e., tile_group_reshaper_model_present_flag (orslice_reshaper_model_present_flag) may be parsed. The purpose of thetile_group_reshaper_model_present_flag (orslice_reshaper_model_present_flag) may be to indicate the presence ofthe reshaping model. For example, whentile_group_reshaper_model_present_flag (orslice_reshaper_model_present_flag) is true (or 1), it may be indicatedthat the reshaper is present for the current tile group (or currentslice). When tile_group_reshaper_model_present_flag (orslice_reshaper_model_present_flag) is false (or 0), it may be indicatedthat the reshaper is not present for the current tile group (or currentslice).

If the reshaper is present and the reshaper is enabled in the currenttile group (or current slice), the reshaper model (i.e.,tile_group_reshaper_model( ) or slice_reshaper_model( ) may beprocessed. Further to this, an additional flag,tile_group_reshaper_enable_flag (or slice_reshaper_enable_flag) may alsobe parsed. The tile_group_reshaper_enable_flag (orslice_reshaper_enable_flag) may indicate whether the reshaping model isused for the current tile group (or slice). For example, iftile_group_reshaper_enable_flag (or slice_reshaper_enable_flag) is 0 (orfalse), it may be indicated that the reshaping model is not used for thecurrent tile group (or the current slice). Iftile_group_reshaper_enable_flag (or slice_reshaper_enable_flag) is 1 (ortrue), it may be indicated that the reshaping model is used for thecurrent tile group (or slice).

As one example, tile_group_reshaper_model_present_flag (orslice_reshaper_model_present_flag) may be true (or 1) andtile_group_reshaper_enable_flag (or slice_reshaper_enable_flag) may befalse (or 0). This means that the reshaping model is present but notused in the current tile group (or slice). In this case, the reshapingmodel can be used in the future tile groups (or slices). As anotherexample, tile_group_reshaper_enable_flag may be true (or 1) andtile_group_reshaper_model_present_flag may be false (or 0). In such acase, the decoder uses the reshaper from the previous initialization.

When the reshaping model (i.e., tile_group_reshaper_model( ) orslice_reshaper_model( ) and tile_group_reshaper_enable_flag (orslice_reshaper_enable_flag) are parsed, it may be determined (evaluated)whether conditions necessary for chroma scaling are present. The aboveconditions includes a condition 1 (the current tile group/slice has notbeen intra-coded) and/or a condition 2 (the current tile group/slice hasnot been partitioned into two separate coding quad tree structures forluma and chroma, i.e. the block structure for The current tilegroup/slice is not a dual tree structure). If the condition 1 and/or thecondition 2 are true and/or tile_group_reshaper_enable_flag (orslice_reshaper_enable_flag) is true (or 1), thentile_group_reshaper_chroma_residual_scale_flag (orslice_reshaper_chroma_residual_scale_flag) may be parsed. Whentile_group_reshaper_chroma_residual_scale_flag (orslice_reshaper_chroma_residual_scale_flag) is enabled (if 1 or true), itmay be indicated that chroma residual scaling is enabled for the currenttile group (or slice). Whentile_group_reshaper_chroma_residual_scale_flag (orslice_reshaper_chroma_residual_scale_flag) is disabled (if 0 or false),it may be indicated that chroma residual scaling is disabled for thecurrent tile group (or slice).

The purpose of the tile group reshaping model is to parse the data thatwould be necessary to construct the lookup tables (LUTs). These LUTs areconstructed on the idea that the distribution of an allowable range ofluma values can be divided into a plurality of bins (ex. 16 bins) whichcan be represented using a set of 16 PWL system of equations. Therefore,any luma value that lies within a given bin can be mapped to an alteredluma value.

FIG. 10 shows a graph representing an exemplary forward mapping. In FIG.10 , five bins are illustrated exemplrily.

Referring to FIG. 10 , the x-axis represents input luma values, and they-axis represents altered output luma values. The x-axis is divided into5 bins or slices, each bin of length L. That is, the five bins mapped tothe altered luma values have the same length. The forward lookup table(FwdLUT) may be constructed using data (i.e., reshaper data) availablefrom the tile group header, and thus mapping may be facilitated.

In one embodiment, output pivot points associated with the bin indicesmay be calculated. The output pivot points may set (mark) the minimumand maximum boundaries of the output range of the luma codewordreshaping. The calculation process of the output pivot points may beperformed by computing a piecewise cumulative distribution function(CDF) of the number of codewords. The output pivot range may be slicedbased on the maximum number of bins to be used and the size of thelookup table (FwdLUT or InvLUT). As one example, the output pivot rangemay be sliced based on a product between the maximum number of bins andthe size of the lookup table (size of LUT*maximum number of binindices). For example, if the product between the maximum number of binsand the size of the lookup table is 1024, the output pivot range may besliced into 1024 entries. This serration of the output pivot range maybe performed (applied or achieved) based on (using) a scaling factor. Inone example, the scaling factor may be derived based on Equation 1below.SF=(y2−y1)*(1<<FP_PREC)+c  [Equation 1]

In Equation 1, SF denotes a scaling factor, and y1 and y2 denote outputpivot points corresponding to each bin. Also, FP_PREC and c may bepredetermined constants. The scaling factor determined based on Equation1 may be referred to as a scaling factor for forward reshaping.

In another embodiment, with respect to inverse reshaping (inversemapping), for a defined range of the bins to be used (i.e., fromreshaper_model_min_bin_idx to reshape_model_max_bin_idx), the inputreshaped pivot points which correspond to the mapped pivot points of theforward LUT and the mapped inverse output pivot points (given by binindex under consideration*number of initial codewords) are fetched. Inanother example, the scaling factor SF may be derived based on Equation2 below.SF=(y2−y1)*(1<<FP_PREC)/(x2−x1)  [Equation 2]

In Equation 2, SF denotes a scaling factor, x1 and x2 denote input pivotpoints, and y1 and y2 denote output pivot points corresponding to eachpiece (bin) (output pivot points of the inverse mapping). Here, theinput pivot points may be pivot points mapped based on a forward lookuptable (FwdLUT), and the output pivot points may be pivot pointsinverse-mapped based on an inverse lookup table (InvLUT). Also, FP_PRECmay be a predetermined constant value. FP_PREC of Equation 2 may be thesame as or different from FP_PREC of Equation 1. The scaling factordetermined based on Equation 2 may be referred to as a scaling factorfor inverse reshaping. During inverse reshaping, partitioning of inputpivot points may be performed based on the scaling factor of Equation 2.The scaling factor SF is used to slice the range of input pivot points.Based on the partitioned input pivot points, bin indices in the rangefrom 0 to the minimum bin index (reshaper_model_min_bin_idx) and/or fromthe minimum bin index (reshaper_model_min_bin_idx) to the maximum binindex (reshape_model_max_bin_idx) are assigned the pivot values thatcorrespond to the minimum and maximum bin values.

In one example, the LMCS data (lmcs_data) may be included in the APS.The semantics of APS may be, for example, 32 APSs signaled for coding.

The following tables show syntax and semantics of an exemplary APSaccording to an embodiment of this document.

TABLE 15 Descriptor adaptation_parameter_set_rbsp( ) { adaptation_parameter_set_id  u(5)  aps_params_type  u(3)  if(aps_params_type = = ALF_APS )   alf_data( adaptation_parameter_set_id ) else if( aps_params_type = = LMCS_APS )   lmcs_data( ) aps_extensicon_flag  u( 1)  if( aps_extension_flag )   while(more_rbsp_data( ) )    aps_extension_data_flag  u(1) rbsp_trailing_bits( ) }

Table 16 is shown in FIG. 21 .

Referring to Table 15, type information of APS parameters (e.g.,aps_params_type) may be parsed/signaled in the APS. Type information ofAPS parameters may be parsed/signaled after adaptation_parameter_set_id.

The aps_params_type, ALF_APS, and LMCS_APS included in Table 15 abovemay be described according to Table 3.2 included in Table 16. That is,according to the aps_params_type included in Table 15, the types of APSparameters applied to the APS may be set as shown in Table 3.2 includedin Table 16. The syntax elements included in Table 15 may be describedwith reference to Table 8. The description related to the APS may besupported by the description provided above together with Tables 1 to 8.

Referring to Table 16, for example, aps_params_type may be a syntaxelement for classifying types of corresponding APS parameters. The typeof APS parameters may include ALF parameters and LMCS parameters.Referring to Table 16, when the value of type information(aps_params_type) is 0, the name of aps_params_type may be determined asALF_APS (or ALF_APS), and the type of APS parameters may be determinedas ALF parameters (the APS parameter may represent the ALF parameters).In this case, the ALF data field (i.e., alf_data( ) may beparsed/signaled to the APS. When the value of type information(aps_params_type) is 1, the name of aps_params_type may be determined asLMCS_APS (or LMCS APS), and the type of APS parameters may be determinedas LMCS parameters (the APS parameter may represent the LMCSparameters). In this case, the LMCS data field (i.e., lmcs_data( ) maybe parsed/signaled to the APS.

Table 17 and/or Table 18 below show(s) syntax of a reshaper modelaccording to an embodiment. The reshaper model may be referred to as anLMCS model. While here the reshaper model has been exemplarily describedas a tile group reshaper, the present specification is not necessarilylimited by this embodiment. For example, the reshaper model may beincluded in the APS, or the tile group reshaper model may be referred toas a slice reshaper model or LMCS data (LMCS data field). Additionally,the prefix “reshaper_model” or “Rsp” may be used interchangeably with“lmcs”. For example, in the following tables and descriptions below,reshaper_model_min_bin_idx, reshaper_model_delta_max_bin_idx,reshaper_model_max_bin_idx, RspCW, and RsepDeltaCW may be usedinterchangeably with lmcs_min_bin_idx, lmcs_delta_cs_bin_idx, lmx mixed,lmcs_delta_csDcs_bin_idx, and Wlm_idx, respectively.

The LMCS data (lmcs_data( ) or the reshaper model (tile group reshaperor slice reshaper) included in Table 15 above may be represented assyntaxes included in the following tables.

TABLE 17 Descriptor tile group reshaper model ( ) { reshaper_model_min_bin_idx ue(v)  reshaper_model_delta_max_bin_idxue(v)  reshaper_model_bin_delta_abs_cw_prec_minus1 ue(v)  for ( i =reshaper_model_min_bin_idx; i<= reshaper_model_max_bin_idx; i−+ ) {  reshape_model_bin_delta_abs_CW [ i ] u(v)   if(reshaper_model_bin_delta_abs_CW[ i ] ) > 0 )   reshaper_model_bin_delta_sign_CW_flag[ i ] u(1)  } }

TABLE 18 Descriptor lmcs_data( )  lmcs_min_bin_idx ue(v) lmcs_delta_max_bin_idx ue(v)  lmcs_delta_cw_prec_minus1 ue(v)  for ( i= lmcs_min_bin_idx; i <= LmcsMaxBinIdx; i++ ) {   lmcs_delta_abs_cw [ i] u(v)   if ( lmcs_delta_abs_cw[ i ] ) > 0 )    lmcs_delta_sign_CW_flag[i ] u(1)  }  ... }

The semantics of syntax elements included in the syntax of Table 17and/or Table 18 may include, for example, matters disclosed in thefollowing table.

TABLE 19 reshape_model_min_bin_idx specifies the minimum bin (or piece)index to be used in the reshaper construction process. The value ofreshape_model_min_bin_idx shall be in the range of 0 to MaxBinIdx,inclusive. The value of MaxBinIdx shall be equal to 15.reshape_model_delta_max_bin_idx specifies the maximum allowed bin (orpiece) index MaxBinIdx minus the maximum bin index to be used in thereshaper construction process. The value of reshape_model_max_bin_idx isset equal to MaxBinIdx − reshape_model_delta_max_bin_idx.reshaper_model_bin_delta_abs_cw_prec_minus1 plus 1 specifies the numberof bits used for the representation of the syntaxreshape_model_bin_delta_abs_CW[ i ]. reshape_model_bin_delta_abs_CW[ i ]specifies the absolute delta codeword value for the ith bin.reshaper_model_bin_delta_sign_CW_flag[ i ] specifies the sign ofreshape_model_bin_delta_abs_CW[ i ] as follows: − If reshapemodel_bin_delta_sign_CW_flag[ i ] is equal to 0, the correspondingvariable  RspDeltaCW[ i ] is a positive value. − Otherwise (reshape_model_bin_delta_sign_CW_flag[ i ] is not equal to 0 ), the corresponding variable RspDeltaCW[ i ] is a negative value. Thevariable OrgCW is derived as follows:  OrgCW = (1 << BitDepthY ) / 16When reshape_model_bin_delta_sign_CW_flag[ i ] is not present, it isinferred to be equal to 0. The variable RspDeltaCW[ i ] = (1 -2*reshape_model_bin_delta_sign_CW [ i ]) *reshape_model_bin_delta_abs_CW [ i ]; The variable RspCW[ i ] is derivedas following steps: The variable OrgCW is set equal to (1 <<BitDepth_(Y) ) / ( MaxBinIdx + 1). − If reshaper_model_min_bin_idx < = i<= reshaper_model_max_bin_idx RspCW[ i ] = OrgCW + RspDeltaCW[ i ]. −Otherwise, RspCW[ i ] = 0. The value of RspCW[ i ] shall be in the rangeof (OrgCW>>3) to (OrgCW<<3 − 1), inclusive. The variables InputPivot[ i] with i in the range of 0 to MaxBinIdx + 1, inclusive are derived asfollows:  InputPivot[ i ] = i * OrgCW The variable ReshapePivot[ i ]with i in the range of 0 to MaxBinIdx + 1, inclusive, the variableScaleCoef[ i ] and InvScaleCoeff[ i ]with i in the range of 0 toMaxBinIdx , inclusive, are derived as follows:  shiftY = 11 ReshapePivot[ 0 ] = 0;  for( i = 0; i <= MaxBinIdx ; i++) {  ReshapePivot[ i + 1 ] = ReshapePivot[ i ] + RspCW[ i ]    ScaleCoef[ i] = ( RspCW[ i ] * (1 << shiftY) + (1 << (Log2(OrgCW) − 1))>> (Log2(OrgCW))    if ( RspCW[ i ] == 0 )     InvScaleCoeff[ i ] = 0   else     InvScaleCoeff[ i ] = OrgCW * (1 << shiftY) / RspCW[ i ]  }The variable ChromaScaleCoef[ i ] with i in the range of 0 to MaxBinIdx, inclusive, are derived as follows: if ( lmcsCW[ i ] = = 0 )   ChromaScaleCoeff[ i ] = (1 << 11) else    ChromaScaleCoeff[ i ] =InvScaleCoeff[ i ]

TABLE 20 lmcs_min_bin_idx specifies the minimum bin (or piece) index tobe used in the luma mapping with chroma scaling construction process.The value of LmcsMaxBinIdx shall be in the range of 0 to MaxBinIdx,inclusive. The value of MaxBinIdx shall be equal to 15.Imcs_delta_max_bin_idx specifies the maximum allowed bin (or piece)index MaxBinIdx minus the maximum bin index to be used in the lumamapping with chroma scaling construction process. The value ofLmcsMaxBinIdx is set equal to 15− lmcs_delta_max_bin_idx.lmcs_delta_cw_prec_minus1 plus 1 specifies the number of bits used forthe representation of the syntax lmcs_delta_abs_cw[ i ].lmcs_delta_abs_cw[ i ] specifies the absolute delta codeword value forthe ith bin. lmcs_delta_sign_sw_flag[ i ] specifies the sign oflmcsDeltaCW[ i ] as follows: − If lmcs_delta_sign_cw_flag[ i ] is equalto 0, the corresponding variable lmcsDeltaCW  CW[ i ] is a positivevalue. − Otherwise ( lmcs_delta_sign_cw_flag[ i ] is not equal to 0),the corresponding variable  lmcsDeltaCW[ i ] is a negative value. Whenlmcs_delta_sign_CW_flag[ i ] is not present, it is inferred to be equalto 0. The variable OrgCW is derived as follows:  OrgCW = (1 << BitDepthY) / 16 The variable lmcsDeltaCW[ i ] with i =lmcs_min_bin_idx..LmcsMaxBinIdx, is derived as follows:  lmcsDeltaCW[i ]= (1 - 2*lmcs_delta_sign_cw_flag [ i ]) * lmcs_delta_abs_cw [ i ]; Thevariable lmcsCW[ i ] is derived as following steps: − Iflmcs_min_bin_idx < = i <= LmcsMaxBinIdx  lmcsCW[ i] = OrgCW +lmcsDeltaCW[i ] − Otherwise, lmcsCW[ i ] = 0. The value of lmcsCW[ i ]shall be in the range of (OrgCW>>3) to (OrgCW<<3 − 1), inclusive. Thevariables InputPivot[ i ] with i in the range of 0 to MaxBinIdx + 1,inclusive are derived as follows:  InputPivot[ i ] = i * OrgCW Thevariable LmcsPivot[ i ] with i in the range of 0 to MaxBinIdx + 1,inclusive, the variable ScaleCoeff[ i ] and InvScaleCoeff[ i ] with i inthe range of 0 to MaxBinIdx , inclusive, are derived as follows:  shiftY= 11  ReshapePivot[ 0 ] = 0;  for( i = 0; i <= MaxBinIdx ; i++) {  ReshapePivot[ i + 1 ] = ReshapePivot[ i ] + RspCW[ i ]    ScaleCoef[ i] = ( RspCW[ i ] * (1 << shiftY) + (1 << (Log2(OrgCW) − 1))) >> (Log2(OrgCW))    if ( RspCW[ i ] == 0 )     InvScaleCoeff[ i ] = 0   else     InvScaleCoeff[ i ] = OrgCW * (1 << shiftY) / RspCW[ i ]  }The variable ChromaScaleCoeff[ i ] with i in the range of 0 to MaxBinIdxinclusive, are derived as follows: if ( lmcsCW[ ] = = 0 )  ChromaScaleCoeff[ i ] = (1 << 11) else (7 91)   ChromaScaleCoeff[ i ]= InvScaleCoeff[ i ]

The inverse mapping process for the luma sample according to the presentdocument may be described in a form of the standard document as shown inthe table below.

TABLE 21 Inverse mapping process for a luma sample Input to this processis a luma sample lumaSample. Output of this process is a modified lumasample invLumaSample. The value of invLumaSample is derived as follows:− If slice_lmcs_enabled_flag of the slice that contains the luma samplelumaSample is equal  to 1, the following ordered steps apply.  1. Thevariable idxYInv is derived by invoking the identification of piece-wisefunction  index process for a luma sample as specified in clause 8.8.2.3with lumaSample as the input  and idxYInv as the output  2. The variableinvSample is derived as follows:   invSample = InputPivot[ idxYInv ] + (InvScaleCoeff[ idxYInv ] *    ( lumaSample = LmcsPivot[ idxYInv ]) + ( 1<< 10 ) ) >> 11  3. The inverse mapped luma sample invLumaSample isderived as follows:   invLumaSample = Clip1Y( invSample ) − Otherwise,invLumaSample is set equal to lumaSample.

Identification of a piecewise function index process for a luma sampleaccording to the present document may be described in a form of thestandard document as shown in the table below. In Table 22, idxYInv maybe referred to as an inverse mapping index, and the inverse mappingindex may be derived based on resconstructed luma samples (lumaSample).

TABLE 22 Identification of piecewise function index process for a lumasample Input to this process is a luma sample lumaSample. Output of thisprocess is an index idxYInv identifing the piece to which the lumasample lumaSample belongs. The variable idxYInv is derived as follows:if ( lumaSample < LmcsPivot[ lmcs_min_bin_idx + 1 ] )  idxYInv =lmcs_min_bin_idx else if ( lumaSample >= LmcsPivot[ LmcsMaxBinIdx ]) idxYInv = LmcsMaxBinIdx else {  for( idxYInv = lmcs_min_bin_idx;idxYInv < LmcsMaxBinIdx; idxYInv++ ) {   if( lumaSample < LmcsPivot [idxYInv + 1 ] )    break  } }

Luma mapping may be performed based on the above-described embodimentsand examples, and the above-described syntax and components includedtherein may be merely exemplary representations, and embodiments in thepresent document are not limited by the above-mentioned tables orequations. Hereinafter, a method for performing chroma residual scaling(scaling for chroma components of residual samples) based on lumamapping is described.

The (luma-dependent) Chroma residual scaling is designed to compensatefor the interaction between the luma signal and its corresponding chromasignals. For example, whether chroma residual scaling is enabled or notis also signalled at the tile group level. In one example, if lumamapping is enabled and if dual tree partition (also known as separatechroma tree) is not applied to the current tile group, an additionalflag is signalled to indicate if the luma-dependent chroma residualscaling is enabled or not. In other example, when luma mapping is notused, or when dual tree partition is used in the current tile group,luma-dependent chroma residual scaling is disabled. In another example,the luma-dependent chroma residual scaling is always disabled for thechroma blocks whose area is less than or equal to 4.

The chroma residual scaling may be based on an average value of acorresponding luma prediction block (a luma component of a predictionblock to which an intra prediction mode and/or an inter prediction modeis applied). Scaling operations at the encoder end and/or the decoderside may be implemented with fixed-point integer arithmetic based onEquation 3 below.c′=sign(c)*((abs(c)*s+2CSCALE_FP_PREC−1)>>CSCALE_FP_PREC)  [Equation 3]

In Equation 3, c′ denotes a scaled chroma residual sample (scaled chromacomponent of a residual sample), c denotes a chroma residual sample(chroma residual sample, chroma component of residual sample), s denotesa chroma residual scaling factor, and CSCALE_FP_PREC denotes a(predefined) constant value to specify precision. For example,CSCALE_FP_PREC may be 11.

FIG. 11 is a flowchart illustrating a method for deriving a chromaresidual scaling index according to an embodiment of the presentdocument. The method in FIG. 11 may be performed based on FIG. 8 , andtables, equations, variables, arrays, and functions included in thedescription related to FIG. 8 .

In the step S1110, it may be determined whether the prediction mode forthe current block is the intra prediction mode or the inter predictionmode based on the prediction mode information. If the prediction mode isthe intra prediction mode, the current block or prediction samples ofthe current block are considered to be already in the reshaped (mapped)region. If the prediction mode is the inter prediction mode, the currentblock or the prediction samples of the current block are considered tobe in the original (unmapped, non-reshaped) region.

In the step S1120, when the prediction mode is the intra predictionmode, an average of the current block (or luma prediction samples of thecurrent block) may be calculated (derived). That is, the average of thecurrent block in the already reshaped area is calculated directly. Theaverage may also be referred to as an average value, a mean, or a meanvalue.

In the step S1121, when the prediction mode is the inter predictionmode, forward reshaping (forward mapping) may be performed (applied) onthe luma prediction samples of the current block. Through forwardreshaping, luma prediction samples based on the inter prediction modemay be mapped from the original region to the reshaped region. In oneexample, forward reshaping of the luma prediction samples may beperformed based on the reshaping model described with Table 17 and/orTable 18 above.

In the step S1122, an average of the forward reshaped (forward mapped)luma prediction samples may be calculated (derived). That is, anaveraging process for the forward reshaped result may be performed.

In the step S1130, a chroma residual scaling index may be calculated.When the prediction mode is the intra prediction mode, the chromaresidual scaling index may be calculated based on the average of theluma prediction samples. When the prediction mode is the interprediction mode, the chroma residual scaling index may be calculatedbased on an average of forward reshaped luma prediction samples.

In an embodiment, the chroma residual scaling index may be calculatedbased on a for loop syntax. The table below shows an exemplary for loopsyntax for deriving (calculating) the chroma residual scaling index.

TABLE 23   for( idxS = 0, idxFound = 0; idxS <= MaxBinIdx; idxS++ ) { if( (S < ReshapePivot[ idxS + 1 ] )  {   idxFound = 1   break;  } }

In Table 23, idxS represents the chroma residual scaling index, idxFoundrepresents an index identifying whether the chroma residual scalingindex satisfying the condition of the if statement is obtained, Srepresents a predetermined constant value, and MaxBinIdx represents themaximum allowable bin index. ReshapPivot[idxS+1] (in other words,LmcsPivot[idxS+1]) may be derived based on Tables 19 and/or 20 describedabove.

In an embodiment, the chroma residual scaling factor may be derivedbased on the chroma residual scaling index. Equation 4 is an example forderiving the chroma residual scaling factor.s=ChromaScaleCoef[idxS]  [Equation 4]

In Equation 4, s represents the chroma residual scaling factor, andChromaScaleCoef may be a variable (or array) derived based on Tables 19and/or 20 described above.

As described above, the average luma value of the reference samples maybe obtained, and the chroma residual scaling factor may be derived basedon the average luma value. As described above, the chroma componentresidual sample may be scaled based on the chroma residual scalingfactor, and the chroma component reconstruction sample may be generatedbased on the scaled chroma component residual sample.

In one embodiment of the present document, a signaling structure forefficiently applying the above-described LMCS is proposed. According tothis embodiment of the present document, for example, LMCS data may beincluded in HLS (i.e., an APS), and through the header information(i.e., picture header, slice header) that is a lower level of the APS,an LMCS model (reshaper model) may be adaptively derived by signalingthe ID of the APS, which is referred to the header information. The LMCSmodel may be derived based on LMCS parameters. Also, for example, aplurality of APS IDs may be signaled through the header information, andthrough this, different LMCS models may be applied in units of blockswithin the same picture/slice.

In one embodiment according to the present document, a method forefficiently performing an operation required for LMCS is proposed.According to the semantics described above in Table 19 and/or Table 20,a division operation by the piece length lmcsCW[i] (also noted asRspCW[i] in the present document) is required to deriveInvScaleCoeff[i]. The piece length of the inverse mapping may not bepower of 2, that means the division cannot be performed by bit shifting.

For example, calculating InvScaleCoeff may require up to 16 divisionsper slice. According to Table 19 and/or Table 20 described above, for 10bit coding, the range of lmcsCW[i] is from 8 to 511, so to implement thedivision operation by lmcsCW[i] using the LUT, the size of the LUT mustbe 504. Also, for 12 bit coding, the range of lmcsCW[i] is from 32 to2047, so the LUT size needs be 2016 to implement the division operationby lmcsCW[i] using the LUT. That is, division is expensive in hardwareimplementation, therefore it is desirable to avoid division if possible.

In one aspect of this embodiment, lmcsCW[i] may be constrained to bemultiple of a fixed number (or a predetermined number or apre-determined number). Accordingly, a lookup table (LUT) (capacity orsize of the LUT) for the division may be reduced. For example, iflmcsCW[i] becomes multiple of 2, the size of the LUT to replace divisionprocess may be reduced by half.

In another aspect of this embodiment, it is proposed that for codingwith higher internal bit depth coding, on the top of existingconstraints “The value of lmcsCW[i] shall be in the range of (OrgCW>>3)to (OrgCW<<3−1)”, further constrain lmcsCW[i] to be multiple of 1(BitDepthY−10) if coding bit depth is higher than 10. Here, BitDepthYmay be the luma bit depth. Accordingly, the possible number of lmcsCW[i]would not vary with the coding bitdepth, and the size of LUT needed tocalculate the InvScaleCoeff do not increase for higher coding bitdepth.For example, for 12-bit internal coding bitdepth, limit the values oflmcsCW[i] being multiple of 4, then the LUT to replace division processwill be the same as what is to be used for 10-bit coding. This aspectcan be implemented alone, but it can also be implemented in combinationwith the above-mentioned aspect.

In another aspect of this embodiment, lmcsCW[i] may be constrained to anarrower range. For example, lmcsCW[i] may be constrained within therange from (OrgCW>>1) to (OrgCW<<1)−1. Then for 10 bit coding, the rangeof lmcsCW[i] may be [32, 127], and therefore it only needs a LUT havinga size of 96 to calculate InvScaleCoeff.

In another aspect of the present embodiment, lmcsCW[i] may beapproximated to closest numbers being power of 2, and used that in thereshaper design. Accordingly, the division in the inverse mapping can beperformed (and replaced) by bit shifting.

In one embodiment according to the present document, constraint of theLMCS codeword range is proposed. According to Table 8 described above,the values of the LMCS codewords are in the range from (OrgCW>>3) to(OrgCW<<3)−1. This codword range is too wide. It may result in visualartifact issue when there are large differences between RspCW [i] andOrgCW.

According to one embodiment according to the present document, it isproposed to constrain the codeword of the LMCS PWL mapping to a narrowrange. For example, the range of lmcsCW[i] may be in the range(OrgCW>>1) to (OrgCW<<1)−1.

In one embodiment according to the present document, use of a singlechroma residual scaling factor is proposed for chroma residual scalingin LMCS. The existing method for deriving the chroma residual scalingfactor uses the average value of the corresponding luma block andderives the slope of each piece of the inverse luma mapping as thecorresponding scaling factor. In addition, the process to identify thepiecewise index requires the availability of the corresponding lumablock, it results in latency problem. This is not desirable for hardwareimplementation. According to this embodiment of the present document,scaling in a chroma block may not depend on a luma block value, and itmay not be necessary to identify a piecewise index. Therefore, thechroma residual scaling process in the LMCS can be performed without alatency issue.

In one embodiment according to the present document, the single chromascaling factor may be derived from both the encoder and the decoderbased on the luma LMCS information. When the LMCS luma model isreceived, the chroma residual scaling factor may be updated. Forexample, when the LMCS model is updated, the single chroma residualscaling factor may be updated.

The table below shows an example for obtaining the single chroma scalingfactor according to the present embodiment.

TABLE 24 Sum = 0; for( i = lmcs_min_bin_idx ; i <= lmcs_max_bin_idx ;i++ ) {  sum += InvScaleCoeff[ i ] { ChromaScaleCoeff= sum / (lmcs_max_bin_idx- lmcs_min_bin_idx +1);

Referring to Table 24, a single chroma scaling factor (ex.ChromaScaleCoeff or ChromaScaleCoeffSingle) may be obtained by averagingthe inverse luma mapping slopes of all pieces within thelmcs_min_bin_idx and lmcs_max_bin_idx.

FIG. 12 illustrates a linear fitting of pivot points according to anembodiment of the present document. In FIG. 12 , pivot points P1, Ps,and P2 are shown. The following embodiments or examples thereof will bedescribed with FIG. 12 .

In an example of this embodiment, the single chroma scaling factor maybe obtained based on a linear approximation of the luma PWL mappingbetween the pivot points lmcs_min_bin_idx and lmcs_max_bin_idx+1(LmcsMaxBinIdx+1). That is, the inverse slope of the linear mapping maybe used as the chroma residual scaling factor. For example, the linearline 1 of FIG. 12 may be a straight line connecting the pivot points P1and P2. Referring to FIG. 12 , in P1, the input value is x1 and themapped value is 0, and in P2, the input value is x2 and the mapped valueis y2. The inverse slope (inverse scale) of linear line 1 is (x2−x1)/y2,and the single chroma scaling factor ChromaScaleCoeffSingle may becalculated based on the input values and mapped values of pivot pointsP1, P2, and the following equation.ChromaScaleCoeffSingle=(x2−x1)*(1<<CSCALE_FP_PREC)/y2  [Equation 5]

In Equation 5, CSCALE_FP_PREC represents a shift factor, for example,CSCALE_FP_PREC may be a predetermined constant value. In one example,CSCALE_FP_PREC may be 11.

In other example according to this embodiment, referring to FIG. 12 ,the input value at the pivot point Ps is min_bin_idx+1 and the mappedvalue at the pivot point Ps is ys. Accordingly, the inverse slope(inverse scale) of the linear line 1 can be calculated as (xs−x1)/ys,and the single chroma scaling factor ChromaScaleCoeffSingle may becalculated based on the input values and mapped values of the pivotpoints P1, Ps, and the following equation.ChromaScaleCoeffSingle=(xs−x1)*(1<<CSCALE_FP_PREC)/ys  [Equation 6]

In Equation 6, CSCALE_FP_PREC represents a shift factor (a factor forbit shifting), for example, CSCALE_FP_PREC may be a predeterminedconstant value. In one example, CSCALE_FP_PREC may be 11, and bitshifting for inverse scale may be performed based on CSCALE_FP_PREC.

In another example according to this embodiment, the single chromaresidual scaling factor may be derived based on a linear approximationline. An example for deriving a linear approximation line may include alinear connection of pivot points (i.e., lmcs_min_bin_idx,lmcs_maxbin_idx+1). For example, the linear approximation result may berepresented by codewords of PWL mapping. The mapped value y2 at P2 maybe the sum of the codewords of all bins (fragments), and the differencebetween the input value at P2 and the input value at P1 (x2−x1) isOrgCW*(lmcs_max_bin_idx−lmcs min_bin_idx+1) (for OrgCW, see Table 19and/or Table 20 above). The table below shows an example of obtainingthe single chroma scaling factor according to the above-describedembodiment.

TABLE 25 Sum = 0; for( i = lmcs_min_bin_idx ; i <= lmcs_max_bin_idx ;i++ ) {  sum += lmcsCW[ i ] } ChromaScaleCoeffSingle = OrgCW *(lmcs_max_bin_idx- lmcs_ min_bin_idx +1)   * (1 << CSCALE_FP_PREC) / sum;

Referring to Table 25, the single chroma scaling factor (ex.ChromaScaleCoeffSingle) may be obtained from two pivot points (i.e.,lmcs_min_bin_idx, lmcs_max_bin_idx). For example, the inverse slope oflinear mapping may be used as the chroma scaling factor.

In another example of this embodiment, the single chroma scaling factormay be obtained by linear fitting of pivot points to minimize an error(or mean square error) between the linear fitting and the existing PWLmapping. This example may be more accurate than simply connecting thetwo pivot points at lmcs_min_bin_idx and lmcs_max_bin_idx. There aremany ways to find the optimal linear mapping and such example isdescribed below.

In one example, parameters b1 and b0 of a linear fitting equationy=b1*x+b0 for minimizing the sum of least square error may be calculatedbased on Equations 7 and/or 8 below.

$\begin{matrix}{{b\; 1} = \frac{\sum_{1}^{n}{( {x_{i} - \overset{\_}{x}} )( {y_{i} - \overset{\_}{y}} )}}{\sum_{1}^{n}( {x_{i} - \overset{\_}{x}} )^{2}}} & \lbrack {{Equation}\mspace{14mu} 7} \rbrack \\{{b\; 0} = {\overset{\_}{y} - {b_{1}\overset{\_}{x}}}} & \lbrack {{Equation}\mspace{14mu} 8} \rbrack\end{matrix}$

In Equation 7 and 8, x is the original luma values, and y is thereshaped luma values, x and y are the mean of x and y, and x_(i) andy_(i) represent values of the i-th pivot points.

Referring to FIG. 12 , another simple approximation to identify thelinear mapping is given as:

-   -   Get the linear line1 by connecting pivot points of PWL mapping        at lmcs_min_bin_idx and lmcs_max_bin_idx+1, calculated        lmcs_pivots_linear[i] of this linear line at input values of        multiples of OrgW    -   Sum up the differences between the pivot points mapped values        using the linear line1 and using PWL mapping.    -   Get the average difference avgDiff.    -   Adjust the last pivot point of the linear line according to the        average difference, e.g., 2*avgDiff    -   Use the inverse slope of the adjusted linear line as the chroma        residual scale.

According to the above-described linear fitting, the chroma scalingfactor (i.e., the inverse slope of forward mapping) may be derived(obtained) based on Equation 9 or 10 below.ChromaScaleCoeffSingle=OrgCW*(1<<CSCALE_FP_PREC)/lmcs_pivots_linear[lmcs_min_bin_idx+1]  [Equation9]ChromaScaleCoeffSingle=OrgCW*(lmcs_max_bin_idx−lmcs_max_bin_idx+1)*(1<<CSCALE_FP_PREC)/lmcs_pivots_linear[lmcs_max_bin_idx+1]  [Equation10]

In the above-described equations, lmcs_pivots_linear[i] may be mappedvalues of linear mapping. With linear mapping, all pieces of PWL mappingbetween minimum and maximum bin indices may have the same LMCS codeword(lmcsCW). That is, lmcs_pivots_linear[lmcs_min_bin_idx+1] may be thesame as lmcsCW[lmcs_min_bin_idx].

Also, in Equations 9 and 10, CSCALE_FP_PREC represents a shift factor (afactor for bit shifting), for example, CSCALE_FP_PREC may be apredetermined constant value. In one example, CSCALE_FP_PREC may be 11.

With the single chroma residual scaling factor (ChromaScaleCoeffSingle),there is no need to calculate the average of the corresponding lumablock, find the index in the PWL linear mapping to get the chromaresidual scaling factor any more. Accordingly, the efficiency of codingusing chroma residual scaling may be increased. This not only eliminatesthe dependency on corresponding luma block, solves the latency problem,but also reduces the complexity.

The luma dependent chroma residual scaling procedure for semanticsand/or chroma samples related to LMCS data according to the presentembodiment described above may be described in a standard documentformat as shown in the following tables.

TABLE 26 7.4.6.4 Luma mapping with chroma scaling data semanticslmcs_min_bin_idx specifies the minimum bin index used in the lumamapping with chroma scaling construction process. The value oflmc_min_bin_idx shall be in the range of 0 to 15, inclusive.lmcs_delta_max_bin_idx specifies the delta value, between 15 and themaxinium bin index LmcsMaxBinIdx used in the luma mapping with chromascaling construction process. The value of lmcs_delta_max_bin_idx shallbe in the range of 0 to 15, inclusive. The value of LmcsMaxBinIdx is setequal to 15 − lmcs_delta_max_bin_idx. The value of LmcsMaxBinIdx shallbe greater than or equal to lmcs_min_bin_idx. lmcs_delta_cw_prec_minus1plus 1 specifics the number of bits used for the representation of thesyntax lmcs_delta_abs_cw[ i ]. The value of lmcs_delta_cw_prec_minus1shall be in the range of 0 to BitDepthY − 2, inclusive. ... The variablediffMaxMinBinIdX, lmcsCWLinear and ChromaScaleCoeffSingle arc derived asfollows  diffMaxMinBinIdX= LmcsMaxBinIdx- lmcs_min_bin_idx+1; lmcsCWLinear = (LmcsPivot[LmcsMaxBinIdx+1]+( diffMaxMinBinIdX>>1))/diffMaxMinBinIdX;  ChrormaScaleCoeffSingle= OrgCW * (1 << 11)/lmcsCWLinear (7-94)

TABLE 27 8.7.5.3 Picture reconstruction with luma dependent chromaresidual scaling process for chroma samples Inputs to this process are:− a location ( xCurr, yCurr ) of the top left sample of the currenttransform block relative to the top left sample  of the current picture,− a variable nCurrSw specifying the transform block width, − a variablenCurrSh specifying the transform block height, − a variable tuCbfChromaspecifying the coded block flag of the current chroma transform block, −an (nCurrSw)x(nCurrSh) array predSamples specifying the chromaprediction samples of the current block, − an (nCurrSw)x(nCurrSh) arrayresSamples specifying the chroma residual samples of the current block.Output of this process is a reconstructed chroma picture sample arrayrecSamples. The reconstructed chroma picture sample recSamples isderived as follows for i = 0..nCurrSw − 1, j = 0..nCurrSh − 1: − If oneof the following conditions is true, recSamples] xCurr + i ][ yCurr + i] is set equal to  Clip1_(C)( predSamples[ i ][ j ] + resSamples[ i ][ j] ): − slice_chroma_residual_scale_flag is equal to 0 − nCurrSw *nCurrSh is less than or euqal to 4 − tu_cbf_cb [ xCurr ][ yCurr ] isequal to 0 and tu_cbf_cr [ xCurr ][ yCurr ] is equal to 0 − Otherwise,the recSamples is derived as follows:   − If tuCbfChroma is equal to 1,the following applies:     resSamples[ i ][ j ] =     Clip3( −( 1 <<BitDepth_(C) ), ( 1 << BitDepth_(C) ) − 1, resSamples[ i ][ j ] )(8-998)     recSamples[ xCurr + i ][ yCurr + j ] = Clip1_(C)(predSamples[ i ][i ] + (8-999)      Sign( resSamples[ i ][ j ] ) * ( (Abs( resSamples[ i ][ j ] ) * ChromaScaleCoeffSingle + ( 1     << 10 )) >> 11 ) )   − Otherwise (tu_cbf is equal to 0), the following applies:    recSamples[ Curr + i ][ yCurr + j ] = Clip1_(C)(predSamples[ i ][ j] ) (8-1000)

In another embodiment of the present document, the encoder may determineparameters related to the single chroma scaling factor and signal theparameters to the decoder. With signaling, the encoder may utitlizeother information available at encoder to derive the Chroma scalingfactor. This embodiment aims to eliminate the chroma residual scalinglatency problem.

For example, another example to identify the linear mapping to be usedfor determine Chorom residual scaling factor is given as below:

-   -   by connecting pivot points of PWL mapping at lmcs_min_bin_idx        and lmcs_max_bin_idx+1, calculated lmcs_pivots_linear[i] of this        linear line at input values of multiples of OrgW    -   Get weighted sum of the differences between the mapped values of        pivot points using the linear line1 and those of luma PWL        mapping. The weight may be based on the encoder statistics such        as histogram of a bin.    -   Get the weighted average difference avgDiff.    -   Adjust the last pivot point of the linear line1 according to the        weighted average difference, e.g., 2*avgDiff    -   Use the inverse slope of the adjusted linear line to calculate        the chroma residual scale.

The tables below shows examples of syntaxes for signaling the y valuefor chroma scaling factor derivation.

TABLE 28 Descriptor lmcs_data ( ) {  lmcs_min_bin_idx ue(v) lmcs_delta_max_bin_idx ue(v)  lmcs_delta_cw_prec_minus1 ue(v)  for ( i= lmcs_min_bin_idx; 1 <= LmcsMaxBinIdx; i++ ) {   lmcs_delta_abs_cw[ i ]u(v)   if ( lmcs_delta_abs_cw[ i ] ) > 0 )    lmcs_delta_sign_cw_flag[ i] u(1)  }  lmcs_chroma_scale u(v) }

In Table 28, the syntax element lmcs_chroma_scale may specify a singlechroma (residual) scaling factor used for LMCS chroma residual scaling(ChromaScaleCoeffSingle=lmcs_chroma_scale). That is, information on thechroma residual scaling factor may be directly signaled, and thesignaled information may be derived as the chroma residual scalingfactor. In other words, the value of the signaled information on thechroma residual scaling factor may be (directly) derived as the value ofthe single chroma residual scaling factor. Here, the syntax elementlmcs_chroma_scale may be signaled together with other LMCS data (i.e., asyntax element related to an absolute value and a sign of the codeword,etc.).

Alternatively, the encoder may signal only necessary parameters toderive the chroma residual scaling factor at decoder. In order to derivethe chroma residual scaling factor at the decoder, it needs an inputvalue x and a mapped value y. Since the x value is the bin length, andis a known number, no need to be signalled. After all, only the y valueneeds to be signaled in order to derive the chroma residual scalingfactor. Here, the y value may be a mapped value of any pivot point inthe linear mapping (i.e., mapped values of P2 or Ps in FIG. 12 ).

The following tables show examples of signaling mapped values forderiving the chroma residual scaling factor.

TABLE 29 Descriptor  lmcs_data ( ) {  lmcs_min_bin_idx ue(v) lmcs_delta_max_bin_idx ue(v)  lmcs_delta_cw_prec_minus1 ue(v)  for ( i= lmcs_min_bin_idx; i <= LmcsMaxBinIdx; i++ ) {   lmcs_delta_abs_cw[ i ]u(v)   if ( lmcs_delta_abs_cw[ i ] ) > 0 )    lmcs_delta_sign_cw_flag[ i] u(1)  }   lmcs_cw_linear u(v) }

TABLE 30 Descriptor lmcs_data ( ) {  lmcs_min_bin_idx ue(v ) lmcs_delta_max_bin_idx ue(v)  lmcs_delta_cw_prec_minus1 uw(v)  for ( i= lmcs_min_bin_idx, i <= LmcsMaxBinIdx; i++ ) {   lmcs_delta_abs_cw[ i ]u(v)   if ( lmcs_delta_abs_cw[ i ] ) > 0 )    lmcs_delta_sign_cw_flag[ i] u(1)  }   lmcs_delta_abs_cw_linear u(v)   if (lmcs_delta_abs_cw_linear ) > 0 )    lmcs_delta_sign_cw_linear_flag u(1)}

One of the syntaxes of Tables 29 and 30 described above may be used tosignal the y value at any linear pivot points specified by the encoderand decoder. That is, the encoder and the decoder may derive the y valueusing the same syntax.

First, an embodiment according to Table 29 is described. In Table 29,lmcs_cw_linear may denote a mapped value at Ps or P2. That is, in theembodiment according to Table 29, a fixed number may be signaled throughlmcs_cw_linear.

In an example according to this embodiment, if lmcs_cw_linear denotes amapped value of one bin (ie, lmcs_pivots_linear[lmcs_min_bin_idx+1] inPs of FIG. 12 ), the chroma scaling factor may be derived based on thefollowing equation.ChromaScaleCoeffSingle=OrgCW*(1<<CSCALE_FP_PREC)/lmcs_cw_linear  [Equation11]

In another example according to this embodiment, if lmcs_cw_lineardenotes lmcs_max_bin_idx+1 (i.e. lmcs_pivots_linear[lmcs_max_bin_idx+1]in P2 of FIG. 12 ), the chroma scaling factor may be derived based onthe following equation.ChromaScaleCoeffSingle=OrgCW*(lmcs_max_bin_idx−lmcs_max_bin_idx+1)*(1<<CSCALE_FP_PREC)/lmcs_cw_linear  [Equation12]

In the above-described equations, CSCALE_FP_PREC represents a shiftfactor (a factor for bit shifting), for example, CSCALE_FP_PREC may be apredetermined constant value. In one example, CSCALE_FP_PREC may be 11.

Next, an embodiment according to Table 30 is described. In thisembodiment, lmcs_cw_linear may be signaled as a delta value relative toa fixed number (i.e. lmcs_delta_abs_cw_linear,lmcs_delta_sign_cw_linear_flag). In an example of this embodiment, whenlmcs_cw_linear represents a mapped value inlmcs_pivots_linear[lmcs_min_bin_idx+1] (i.e. Ps of FIG. 12 ),lmcs_cw_linear_delta and lmcs_cw_linear may be derived based on thefollowing equations.lmcs_cw_linear_delta=(1−2*lmcs_delta_sign_cw_linear_flag)*lmcs_delta_abs_linear_cw  [Equation13]lmcs_cw_linear=lmcs_cw_linear_delta+OrgCW  [Equation 14]

In another example of this embodiment, when lmcs_cw_linear represents amapped value in lmcs_pivots_linear[lmcs_max_bin_idx+1] (i.e. P2 of FIG.12 ), lmcs_cw_linear_delta and lmcs_cw_linear may be derived based onthe following equations.lmcs_cw_linear_delta=(1−2*lmcs_delta_sign_cw_linear_flag)*lmcs_delta_abs_linear_cw  [Equation15]lmcs_cw_linear=lmcs_cw_linear_delta+OrgCW*(lmcs_max_bin_idx−lmcs_max_bin_idx+1)  [Equation16]

In the above-described equations, OrgCW may be a value derived based onTable 19 and/or Table 20 described above.

The luma dependent chroma residual scaling procedure for semanticsand/or chroma samples related to LMCS data according to the presentembodiment described above may be described in a standard documentformat as shown in the following tables.

TABLE 31 7.4.6.4 Luma mapping with chroma scaling data semanticslmcs_min_bin_idx specifies the minimum bin index used in the lumamapping with chroma scaling construction process. The value oflmcs_min_bin_idx shall be in the range of 0 to 15, inclusive.lmcs_delta_max_bin_idx specifies the delta value between 15 and themaximum bin index LmcsMaxBinIdx used in the luma mapping with chromascaling construction process. The value of lmcs_delta_max_bin_idx shallbe in the range of 0 to 15, inclusive. The value of LmcsMaxBinIdx is setequal to 15 − lmcs_delta_max_bin_idx the value of LmcsMaxBinIdx shall begreater than or equal to lmcs_min_bin_idx. lmcs_delta_cw_prec_minus1plus 1 specifies the number of bits used for the representation of thesyntax lmcs_delta_abs_cw[ i ] and lmcs_delta_abs_cw_linear. The value oflmcs_delta_cw_prec_minus1 shall be in the range of 0 to BitDepthY − 2,inclusive.  ... lmcs_delta_abs_cw_linear specifies the absolute deltacodeword value for the ith bin. lmcs_delta_sign_cw_linear_flag specifiesthe sign of the variable lmcsDeltaCWLinear as follows: − Iflmcs_delta_sign_cw_linear_flag is equal to 0, lmcsDeltaCWLinear is apositive value. − Otherwise ( lmcs_delta_sign_cw_linear_flag is notequal to 0 ) lmcsDeltaCWLinear is a negative value. Whenlmcs_delta_sign_cw_linear_flag is not present, it is inferred to beequal to 0. The variable lmcsDeltaCWLinear, is derived as follows:lmcsDeltaCWLinear = ( 1 − 2 * lmcs_delta_sign_cw_linear_flag ) *lmcs_delta_abs_cw_linear lmcsCWLinear = OrgCW + lmcsDeltaCWLinear Thevariable ChromaScaleCoeffSingle is derived as followsChromaScaleCoeffSingle = OrgCW * (1 << 11) / lmcsCWLinear

TABLE 32 8.7.5.3 Picture reconstruction with luma dependent chromaresidual scaling process for chroma samples Inputs to this process are:− a location ( xCurr, yCurr ) of the top-left sample of the currenttransform block relative to the top-left sample  of the current picture,− a variable nCurrSw specifying the transform block width, − a variablenCurrSh specifying the transform block height, − a variable tuCbfChromaspecifying the coded block flag of the current chroma transform block, −an (nCurrSw)x(nCurrSh) array predSamples specifying the chromaprediction samples of the current block, − an (nCurrSw)x(nCurrSh) arrayresSamples specifying the chroma residual samples of the current block.Output of this process is a reconstructed chroma picture sample arrayrecSamples. The reconstructed chroma picture simple recSamples isderived as follows for i = 0..nCurrSw − 1, j = 0..nCurrSh − 1: − If oneof the following conditions is true, recSamples[ xCurr + i ][ yCurr + j] is set equal to  Clip1_(C)( predSamples[ i ][ j ] + resSamples[ i ] [j ] ): − slice_chroma_residual_scale_flag is equal to 0 − nCurrSw *nCurrSh is less than or euqal to 4 − tu_cbf_cb [ xCurr ][ yCurr ] isequal to 0 and tu_cbf_cr [ xCurr ][ yCurr ] is equal to 0 − Otherwise,the recSamples is derived as follows:   − If tuCbfChroma is equal to 1,the following applies:    resSamples[ i ][ j ] =    Clip3( −( 1 <<BitDepth_(C) ), ( 1 << BitDepth_(C) ) − 1, resSamples[ i ][ j ] )(8-998)    recSamples[ xCurr + i ] [ yCurr + j ] = Clip1C( predSamples[i ][ j ] + (8-999)     Sign( resSamples[ i ][ j ] ) * ( ( Abs(resSamples[ i ][ j ] ) * ChromaScaleCoeffSingle +    ( 1 << 10 ) ) >> 11) )   − Otherwise (tu_cbf is equal to 0), the following applies:   recSamples[ xCurr + i ][ yCurr + j ] = Clip1_(C)(predSamples[ i ] [ j] ) (8-1000)

FIG. 13 illustrates one example of linear reshaping (or linearreshaping, linear mapping) according to an embodiment of the presentdocument. That is, in this embodiment, the use of a linear reshaper inLMCS is proposed. For example, this example in FIG. 13 may relate toforward linear reshaping (mapping).

Referring to FIG. 13 , the linear reshaper may include two pivot pointsi.e., P1 and P2. P1 and P2 may represent input and mapped values, forexample P1 may be (min_input, 0) and P2 may be (max_input, max_mapped).Here, min_input represents the minimum input value, and max_inputrepresents the maximum input value. Any input value less than or equalto min_input are mapped to 0, any input value larger than max_input aremapped to max_mapped. Any input luma values within the min_input andmax_input are linearly mapped to other values. FIG. 13 shows an exampleof mapping. The pivot points P1, P2 may be determined at the encoder,and a linear fitting may be used to approximate the piecewise linearmapping.

In another embodiment according to the present document, another exampleof a method for signaling the linear reshaper may be proposed. The pivotpoints P1, P2 of the linear reshaper model may be explicitly signaled.The following tables show an example of syntax and semantics forexplicitly signaling the linear reshaper model according to thisexample.

TABLE 33 Descriptor lmcs_data ( ) {  lmcs_min_input ue(v) lmcs_max_input ue(v)  lmcs_max_mapped ue(v)

TABLE 34 lmcs_min_input specifies the input value of the 1st pivotpoint. It has a mapped value of 0. lmcs_max_input is the input value ofthe 2nd pivot point. lmcs_max_mapped is the mapped value at the 2^(nd)pivot points. They may be signaled explicitly using explo golomb code orfixed length code with length depending on BitDepth_(Y). lmcsCWLinearAll = lmcs_max_mapped The variable ScaleCoeffSingle andInvScaleCoeffSingle are derived as follows:  Rounding = (lmcs_max_input-lmcs_min_input)>>1  ScaleCoeffSingle = ( lmcsCWLinearAll * (1 <<FP_PREC) + Rounding))   / (lmcs_max_input-lmcs_min_input); InvScaleCoeffSingle = (lmcs_max_input- lmcs_min_input)*(1<<FP_PREC) /lmcsCWLinearAll The variable ChromaScaleCoeffSingle is derived asfollows: ChromaScaleCoeffSingle = InvScaleCoeffSingle >> (FP_PREC-CSCALE_FP_PREC)

Referring to Tables 33 and 34, the input value of the first pivot pointmay be derived based on the syntax element lmcs_min_input, and the inputvalue of the second pivot point may be derived based on the syntaxelement lmcs_max_input. The mapped value of the first pivot point may bea predetermined value (a value known to both the encoder and decoder),for example, the mapped value of the first pivot point is 0. The mappedvalue of the second pivot point may be derived based on the syntaxelement lmcs_max_mapped. That is, the linear reshaper model may beexplicitly (directly) signaled based on the information signaled in thesyntax of Table 33.

Alternatively, lmcs_max_input and lmcs_max_mapped may be signaled asdelta values. The following tables show an example of syntax andsemantics for signaling a linear reshaper model as delta values.

TABLE 35 Descriptor lmcs_data ( ) {  lmcs_min_input ue(v) lmcs_max_input_delta ue(v)  lmcs_max_mapped_delta ue(v)

TABLE 36 lmcs_max_input_delta specifies the difference between the inputvalue of the 2nd pivot point to the max luma value (1<<bitdepthY)−1, and lmcs_max_input = (1<<bitdepthY)−1 − lmcs_max_input_delta;lmcs_max_mapped_delta specifies the difference between the mapped valueof the 2nd pivot point to the max luma value (1<<bitdepthY)−1. lmcsCWLinearAll = lmcs_max_mapped = (1<<bitdepthY)−1 −lmcs_max_mapped_delta The variable ScaleCoeffSingle andInvScaleCoeffSingle are derived as follows:  Rounding = (lmcs_max_input−lmcs_min_input)>>1  ScaleCoeffSingle = ( lmcsCWLinearAll * (1 <<FP_PREC) + Rounding))   / (lmcs_max_input−lmcs_min_input); InvScaleCoeffSingle = (lmcs_max_input− lmcs_min_input)*(1<<FP_PREC) /lmcsCWLinearAll The variable ChromaScaleCoeffSingle is derived asfollows:  ChromaScaleCoeffSingle = InvScaleCoeffSingle >> (FP_PREC-CSCALE_FP_PREC)

Referring to Table 36, the input value of the first pivot point may bederived based on the syntax element lmcs_min_input. For example,lmcs_min_input may have a mapped value of 0. lmcs_max_input_delta mayspecify the difference between the input value of the second pivot pointand the maximum luma value (i.e., (1<<bitdepthY)−1).lmcs_max_mapped_delta may specify the difference between the mappedvalue of the second pivot point and the maximum luma value (i.e.,(1<<bitdepthY)−1).

According to an embodiment of the present document, forward mapping forluma prediction samples, inverse mapping for luma reconstructionsamples, and chroma residual scaling may be performed based on theabove-described examples of the linear reshaper. In one example, onlyone inverse scaling factor may be needed for inverse scaling for luma(reconstructed) samples (pixels) in linear reshaper based inversemapping. This is also true for forward mapping and chroma residualscaling. That is, the steps to determine ScaleCoeff[i], InvScaleCoeff[i]and ChromaScaleCoeff[i] with i being the bin index, may be replaced withjust one single factor. Here, one single factor is a fixed pointrepresentation of the (forward) slope or the inverse slope of the linearmapping. In one example, the inverse luma mapping scaling factor(inverse scaling factor in inverse mapping for luma reconstructionsamples) may be derived based on at least one of the followingequations.InvScaleCoeffSingle=OrgCW/lmcsCWLinear  [Equation 17]InvScaleCoeffSingle=OrgCW*(lmcs_max_bin_idx−lmcs_max_bin_idx+1)/lmcsCWLinearAll  [Equation 18]InvScaleCoeffSingle=(lmcs_max_input−lmcs_min_input)/lmcsCWLinearAll  [Equation19]

The lmcsCWLinear of Equation 17 may be derived from Table 31 describedabove. lmcsCWLinearALL of Equations 18 and 19 may be derived from atleast one of Tables 33 to 36 described above. In Equation 17 or 18,OrgCW may be derived from Table 19 and/or 20.

The following tables describe equations and syntax (conditionalstatements) indicating a forward mapping process for luma samples (i.e.luma prediction samples) in picture reconstruction. In the followingtables and equations, FP_PREC is a constant value for bit shifting, andmay be a predetermined value. For example, FP_PREC may be 11 or 15.

TABLE 37 idxY = predSamples[ i ][ i ] >> Log2( OrgCW ) PredMapSamples[ i][ j ] = LmcsPivot[ idxY ]  + ( ScaleCoeff[ idxY ] * ( predSamples[ i ][j ] − InputPivot[ idxY ] ) + ( 1 << 10 ) ) >> 11  with i = 0..nCurrSw −1, j = 0..nCurrSh − 1

TABLE 38 if (PredMapSamples[ i ][ j ] <= lmcs_min_input) PredMapSamples[ i ][ j ]=0 else if (PredMapSamples[ i ][ j ] >=lmcs_max_input)  PredMapSamples[ i ][ j ]=lmcs_max_mapped else PredMapSamples[ i ][ j ] = ( ScaleCoeffSingle * predSamples[ i ][ j ]  + ( 1 <<( FP_PREC-1) ) ) >> FP_PREC

Table 37 may be for deriving forward mapped luma samples in the lumamapping process based on Tables 17 to 20 described above. That is, Table37 may be described together with Tables 19 and 20. In Table 37, theforward mapped luma (prediction) samples PredMapPSamples[i][j] as outputcan be derived from luma (prediction) samples predSamples[i][j] asinput. idxY of Table 37 may be referred to as a (forward) mapping index,and the mapping index may be derived based on prediction luma samples.

Table 38 may be for deriving forward mapped luma samples in the linearreshaper based luma mapping. For example, lmcs_min_input,lmcs_max_input, lmcs_max_mapped, and ScaleCoeffSingle of Table 38 may bederived by at least one of Tables 33 to 36. In Table 38, in a case of‘lmcs_min_input<predSamples[i][j]<lmcs_max_input’, forward mapped luma(prediction) samples PredMapSamples[i][j] may be derived as output fromthe input, luma (prediction) samples predSamples[i][j]. In comparisonbetween Table 37 and Table 38, the change from the existing LMCSaccording to the application of the linear reshaper can be seen from theperspective of forward mapping.

The following equations and tables describe an inverse mapping processfor luma samples (i.e. luma reconstruction samples). In the followingequations and tables, ‘lumaSample’ as an input may be a lumareconstruction sample before inverse mapping (before modification).‘invSample’ as output may be an inverse mapped (modified) lumareconstruction sample. In other cases, the clipped invSample may bereferred to as a modified luma reconstruction sample.invSample=InputPivot[idxYInv]+(InvScaleCoeff[idxYInv]*(lumaSample−LmcsPivot[idxYInv])+(1<<(FP_PREC−1)))>>FP_PREC  [Equation20]invSample=lmcs_min_input+(InvScaleCoeffSingle*(lumaSample−lmcs_min_input)+(1<<(FP_PREC−1)))>>FP_PREC  [Equation21]

TABLE 39 Use single inverse scale factor, derive the invserse lumasample as a whole linear piece.  lumaSample = Clip3(0,LmcsPivot[LmcsMaxBinIdx+1], lumaSample)    // Clip3(min, max val) invSample = InputPivot[ lmcs_min_bin_idx ] + ( InvScaleCoeffSingle *  ( lumaSample − LmcsPivot[ lmcs_min_bin_idx ] ) + ( 1 << ( FP_PREC-1) )) >> FP_PREC where Clip3(min, max, val) denotes to clip the input vlaueval within range min and max

TABLE 40 If the pivot point are represented by (lmcs_min_input,0)(lmcs_max_input, lmcs_max_mapped)  lumaSample = Clip3(0,lmcs_max_mapped, lumaSample)    // Clip3(min, max, val) invSample =lmcs_min_input + ( InvScaleCoeffSingle * lumaSample + ( 1 << (FP_PREC-1)) ) >> FP_PREC

Equation 21 may be for deriving inverse mapped luma samples in lumamapping according to this document. In Equation 20, the index idxInv maybe derived based on Tables 50, 51, or 52 to be described later.

Equation 21 may be for deriving inverse mapped luma samples from lumamapping according to the application of the linear reshaper. Forexample, lmcs_min_input of Equation 21 may be derived from at least oneof Tables 33 to 36. Through the comparison between Equation 20 andEquation 21, the change from the existing LMCS according to theapplication of the linear reshaper can be seen from the perspective offorward mapping.

Table 39 may include an example of equations for deriving inverse mappedluma samples in luma mapping. For example, the index idxInv may bederived based on Tables 50, 51, or 52 to be described later.

Table 40 may include other examples of equations for deriving inversemapped luma samples in luma mapping. For example, lmcs_min_input and/orlmcs max_mapped of Table 40 may be derived by at least one of Tables 33to 36, and/or InvScaleCoeffSingle of Table 40 is at least of Tables 33to 36, and/or Equations 17 to 19 can be derived by one.

Based on the above-described examples of the linear reshaper, thepiecewise index identification process may be omitted. That is, in thepresent examples, since there is only one piece having valid reshapedluma pixels, the piecewise index identification process used for inverseluma mapping and chroma residual scaling can be removed. Accordingly,the complexity of inverse luma mapping may be reduced. In addition,latency problems caused by depending on luma piecewise indexidentification during chroma residual scaling can be eliminated.

According to the embodiment of the use of the linear reshaper describedabove, the following advantages may be provided for LMCS: i) It ispossible to simplify the encoder reshaper design, preventing possibleartifact by abrupt changes between the piecewise linear pieces ii) Thedecoder inverse mapping process, in which the piecewise indexidentification process can be removed, can be simplified by eliminatingthe piecewise index identification process iii) By removing thepiecewise index identification process, it is possible to remove thelatency problem in the chroma residual scaling caused by depending onthe corresponding luma blocks iv) It is possible to reduce overhead ofsignaling, and make frequent update of reshaper more feasible v) Formany places that used to require of a loop of 16 pieces, the loop can beelimated. For example, to derive InvScaleCoeff[i], the number ofdivision operations by lmcsCW[i] can be reduced to 1.

In another embodiment according to the present document, an LMCS basedon flexible bins is proposed. Here, the flexible bins may refers to thenumber of bins not fixed to a predetermined (predefined, specific)number. In the existing embodiment, the number of bins in the LMCS isfixed to 16, and the 16 bins are equally distributed for input samplevalues. In this embodiment, the flexible number of bins is proposed andthe pieces (bins) may not be equally distributed in terms of theoriginal pixel values.

The following tables exemplarily show syntax for LMCS data (data field)according to this embodiment and semantics for syntax elements includedtherein.

TABLE 41 Descriptor lmcs_data ( ) {  lmcs_num_bins_minus1 ue(v)   for (i = 0; i <= lmcs_num_bins: i++ ) { u(v)     lmcs_delta_input_cw[ i ]u(v)    lmcs_delta_mapped_cw[ i ] u(v)   }

TABLE 42 lmcs_num_bins_minus1 plus 1 specifies the number of bins.lmcs_num_ bins shall be in the range of 1 and 1<<BitDepth_(Y) −1.lmcs_num_bins or lmcs_num_bins_minus1 may be restricted to multiple ofpower of 2, or log of 2 to reduce the number of bits used for signaling.lmcs_num_bins = lmcs_num_bins_minus1 + 1 lmcs_delta_input_cw[ i ]specifies the delta input value of the ith pivot point relative theprevious pivot point. lmcs_delta_input_cw[ i ] >= 0.lmcs_delta_mapped_cw[ i ] specifies the delta mapped value of the ithpivot point relative the previous pivot point. lmcs_delta_mapped_cw[ i] >= 0 The variable LmcsPivot_input[ i ] and LmcsPivot_mapped[ i ] withi = 0.. lmcs_num_bins+1 LmcsPivot_input[ 0 ] = 0; LmcsPivot_mapped└ 0 ┘= 0; for( i = 0; i <= lmcs_num_bins; i++) {  LmcsPivot_input[ i + 1 ] =LmcsPivot_input[ i ] + lmcs_delta_input_  cw[ i ];  LmcsPivot_mapped[i + 1 ] = LmcsPivot_mapped[ i ] + lmcs_delta_  mapped_cw[ i ]; }LmcsPivot_input[ lmcs_num_bins+1] shall equal to (1 << BitDepth_(Y) ) −1 . I.e. sum of all lmcs_delta_input_ cw[ i ] shall equal to (1 <<BitDepth_(Y) ) − 1, therefore the last lmcs_delta_ mapped_cw[ i ] can beinferred without signaling. LmcsPivot_mapped[ lmcs_num_bins+1], I.e. sumof all lmcs_delta_ mapped_ cw[ i ] shall be not greater than (1 <<BitDepth_(Y) ) − 1.

Referring to Table 41, information on the number of binslmcs_num_bins_minus1 may be signaled. Referring to Table 42,lmcs_num_bins_minus1+1 may be equal to the number of bins, from whichthe number of bins may be within a range from 1 to (1<<BitDepthY)−1. Forexample, lmcs_num_bins_minus1 or lmcs_num_bins_minus1+1 may be multipleof power of 2.

In the embodiment described together with tables 41 and 42, the numberof pivot points can be derived based on lmcs_num_bins_minus1(information on the number of bins), regardless of whether the reshaperis linear or not (signaling of lmcs_num_bins_minus1), and input valuesand mapped values of pivot points (LmcsPivot_input[i],LmcsPivot_mapped[i]) may be derived based on the summation of signaledcodeword values (lmcs_delta_input_cw[i], lmcs_delta_mapped_cw[i]) (here,the initial The input value LmcsPivot input[0] and the initial outputvalue LmcsPivot_mapped[0] are 0).

FIG. 14 shows an example of linear forward mapping in an embodiment ofthe present document. FIG. 15 shows an example of inverse forwardmapping in an embodiment of the present document.

In the embodiment according to FIG. 14 and FIG. 15 , a method forsupporting both regular LMCS and linear LMCS is proposed. In an exampleaccording to this embodiment, a regular LMCS and/or a linear LMCS may beindicated based on the syntax element lmcs_is_linear. In the encoder,after the linear LMCS line is determined, the mapped value (i.e., themapped value in pL of FIG. 14 and FIG. 15 ) can be divided into equalpieces (i.e., LmcsMaxBinIdx−lmcs_min_bin_idx+l). The codeword in the binLmcsMaxBinIdx may be signaled using the syntaxes for the lmcs data orreshaper mode described above.

The following tables exemplarily show syntax for LMCS data (data field)and semantics for syntax elements included therein according to anexample of this embodiment.

TABLE 43 Descriptor lmcs_data ( ) {    lmcs_min_bin_idx ue(v)   lmcs_delta_max_bin_idx ue(v)    lmcs_is_linear_flag u(1)   lmcs_delta_cw_prec_minus1 ue(v)   If (lmcs_is_linear_flag){     lmcs_bin_delta_abs_CW_linear u(v)      if(lmcs_bin_delta_abs_CW_linear) > 0 )      lmcs_bin_delta_sign_CW_linear_flag u(1)     }  else {     for ( i= lmcs_min_bin_idx; i <= LmcsMaxBinIdx;     i++ ) {     lmcs_delta_abs_cw[ i ] u(v)      if ( lmcs_delta_abs_cw[ i ] ) > 0)       lmcs_delta_sign_cw_flag[ i ] u(1)    }  } }

TABLE 44 lmcs_min_bin_idx specifies the minimum bin index used in theluma mapping with chroma scaling construction process. The value oflmcs_min_bin_idx shall be in the range of 0 to 15, inclusive.lmcs_delta_max_bin_idx specifies the delta value between 15 and themaximum bin index LmcsMaxBinIdx used in the luma mapping with chromascaling construction process. The value of lmcs_delta_max_bin_idx shallbe in the range of 0 to 15, inclusive. The value of LmcsMaxBinIdx is setequal to 15 − lmcs_delta_max_bin_idx. The value of LmcsMaxBinIdx shallbe greater than or equal to lmcs_min_bin_idx. lmcs_is_linear_flag equalto 1 specifies that LMCS model is a linear model, otherwise, it is aregular 16-piece PWL model. lmcs_delta_cw_prec_minus1 plus 1 specifiesthe number of bits used for the representation of the syntaxlmcs_delta_abs_cw[ i ] if lmcs_is_linear_flag is false and specifies thenumber of bits used for the representation of thelmcs_bin_delta_abs_CW_linear if lmcs_is_linear_flag is true. The valueof lmcs_delta_cw_prec_minus1 shall be in the range of 0 to BitDepthY −2, inclusive. lmcs_delta_abs_cw_linear specifies the absolute deltacodeword value of one bin of linear mapping.lmcs_delta_sign_cw_linear_flag specifies the sign of the variablelmcsDeltaCWLinear as follows: - If lmcs_delta_sign_cw_linear_flag isequal to 0, lmcsDeltaCWLinear is a positive value. - Otherwise (lmcs_delta_sign_cw_linear flag is not equal to 0) lmcsDeltaCWLinear is a  negative value. When lmcs_delta_sign_cw_linear_flag is not present, itis inferred to be equal to 0. lmcs_delta_abs_cw[ i ] specifies theabsolute delta codeword value for the ith bin. lmcs_delta_sign_cw_flag[i ] specifies the sign of the variable lmcsDeltaCW[ i ] as follows: Iflmcs_delta_sign_cw_flag[ i ] is equal to 0, lmcsDeltaCW[ i ] is apositive value. Otherwise ( lmcs_delta_sign_cw_flag[ i ] is not equal to0 ), lmcsDeltaCW[ i ] is a negative value. When lmcs_delta_sign_cw_flag[i ] is not present, it is inferred to be equal to 0. The variable OrgCWis derived as follows:     OrgCW = (1 << BitDepthY ) / 16               (7-88) The variable lmcsDeltaCW[ i ], with i =lmcs_min_bin_idx..LmcsMaxBinIdx, is derived as follows:  iflmcs_is_linear_flag is true,    lmcsDeltaCW[ i ] =      ( 1 − 2 *lmcs_delta_sign_cw_linear_flag ) * lmcs_delta_abs_cw_linear  else    lmcsDeltaCW[ i ] =     ( 1 − 2 * lmcs_delta_sign_cw_flag[ i ] ) *lmcs_delta_abs_cw[ i ]    (7-89) The variable lmcsCW[ i ] is derived asfollows: - For i =0.. lmcs_min_bin_idx − 1, lmcsCW[ i ] is set equal0. - For i = lmcs_min_bin_idx..LmcsMaxBinIdx, the following applies:    lmcsCW[ i ] = OrgCW + lmcsDeltaCW[ i ]             (7-90)   Thevalue of lmcsCW[ i ] shall be in the range of (OrgCW>>3) to (OrgCW<<3 −1),   inclusive. - For i = LmcsMaxBinIdx + 1..15, lmcsCW[ i ] is setequal 0. It is a requirement of bitstream conformance that the followingcondition is true:     Σ_(t=0) ¹⁵ lmcsCW[ i ] <= (1 << BitDepthY ) − 1           (7-91)

The following tables exemplarily show syntax for LMCS data (data field)and semantics for syntax elements included therein according to anotherexample of this embodiment.

TABLE 45 Descriptor lmcs_data ( ) {  lmcs_min_bin_idx ue(v) lmcs_delta_max_bin_idx ue(v)  lmcs_is_linear_flag u(1) lmcs_delta_cw_prec_minus1 ue(v)  for ( i =lmcs_min_bin_idx; i <=LmcsMaxBinEnd; i++ ) {   lmcs_delta_abs_cw[ i ] u(v)   if (lmcs_delta_abs_cw[ i ] ) > 0 )    lmcs_delta_sign_cw_flag[ i ] u(1)  } }

TABLE 46 lmcs_min_bin_idx specifies the minimum bin index used in theluma mapping with chroma scaling construction process. The value oflmcs_min_bin_idx shall be in the range of 0 to 15, inclusive.lmcs_delta_max_bin_idx specifies the delta value between 15 and themaximum bin index LmcsMaxBinIdx used in the luma mapping with chromascaling construction process. The value of lmcs_delta_max_bin_idx shallbe in the range of 0 to 15, inclusive. The value of LmcsMaxBinIdx is setequal to 15 − lmcs_delta_max_bin_idx. The value of LmcsMaxBinIdx shallbe greater than or equal to lmcs_min_bin_idx. lmcs_is_linear_flag equalto 1 specifies that LMCS model is linear model, otherwise, equal to 0specifies that LMCS model is a regular 16-piece PWL model.lmcs_delta_cw_prec_minus1 plus 1 specifies the number of bits used forthe representation of the syntax lmcs_delta_abs_cw[ i ]. The value oflmcs_delta_cw_prec_minus1 shall be in the range of 0 to BitDepthY − 2,inclusive. Variables LmcsMaxBinEnd is derived as below:      if(lmcs_is_linear_flag==1)       LmcsMaxBinEnd = lmcs_min_bin_idx     else       LmcsMaxBinEnd = LmcsMaxBinIdx lmcs_delta_abs_cw[ i ]specifies the absolute delta codeword value for the ith bin.lmcs_delta_sign_cw_flag[ i ] specifies the sign of the variablelmcsDeltaCW[ i ] as follows: - If lmcs_delta_sign_cw_flag[ i ] is equalto 0, lmcsDeltaCW[ i ] is a positive value. - Otherwise (lmcs_delta_sign_cw_flag[ i ] is not equal to 0 ) lmcsDeltaCW[ i ] is anegative   value. When lmcs_delta_sign_cw_flag[ i ] is not present, itis inferred to be equal to 0. When lmcs_is_linear flag is true, thestart index and end index of the loop to receive lmcs_delta_abs_cw[ i ]are the same, therefore the loop is only entered once, only one set oflmcs_delta_abs_cw[ i ] and lmcs_delta_sign_cw[i] is received, and isstored in lmcs_delta_abs_cw[lmcs_min_bin_idx ] andlmcs_delta_sign_cw[lmcs_min_bin_idx]. The variable OrgCW is derived asfollows:     OrgCW = (1 << BitDepthY ) / 16               (7-88) Thevariable lmcsDeltaCW[ i ], with i = lmcs_min_bin_idx..LmcsMaxBinIdx, isderived as follows: if lmcs_is_linear_flag is true,    lmcsDeltaCW[ i ]=  ( 1 − 2 * lmcs_delta_sign_cw_flag[lmcs_min_bin_idx] ) *lmcs_delta_abs_cw[lmcs_min_  bin_idx]  else lmcsDeltaCW[ i ] = 1 − 2lmcs_delta_sign_cw_flag[ i ] ) * lmcs_delta_abs_cw[ i ] (7-89)

Referring to Tables 43 to 46, when lmcs_is_linear_flag is true, all thelmcsDeltaCW[i] between lmcs_min_bin_idx and LmcsMaxBinIdx may have thesame values. That is, lmcsCW[i] of all pieces between lmcs_min_bin_idxand LmcsMaxBinIdx, may have the same values. The scale and inverse scaleand chroma scale of all pieces between lmcs_min_bin_idx andlmcsMaxBinIdx may be the same. Then if the linear reshaper is true, thenthere is no need to derive the piece index, it can use scale, inversescale from just one of the pieces.

The following table exemplarily shows the identification process of thepiecewise index according to the present embodiment.

TABLE 47 Identification of piecewise function index process for a lumasample  if lmcs_is_linear_flag is true,    idxYInv = lmcs_min_bin_idx else  {   /* Use existing piecewise index identification process */  }_

According to another embodiment of the present document, the applicationof regular 16-piece PWL LMCS and linear LMCS may be dependent on higherlevel syntax (i.e., sequence level).

The following tables exemplarily show the syntax for the SPS accordingto the present embodiment and the semantics for the syntax elementsincluded therein.

TABLE 48 Descriptor seq_parameter_set_rbsp( ) { ....  sps_linear_lmcs_enabled_flag u(1) ...

TABLE 49 sps_linear_lmcs_enabled_flag equal to 0 specifies that thelinear lmcs is disabled and only regular LMCS is enabled.sps_linear_lmcs_ enabled_flag equal to 1 specifies that the linear lmcsis enabled and regular LMCS is disabled.

Referring to Tables 48 and 49, enabling of the regular LMCS and/or thelinear LMCS may be determined (signaled) by a syntax element included inthe SPS. Referring to Table 35, based on the syntax elementsps_linear_lmcs_enabled_flag, one of a regular LMCS or a linear LMCS maybe used in units of a sequence.

In addition, whether to only enable the linear LMCS, or the regular LMCSor both may also be dependent on profile level. In one example, for aspecific profile (i.e., SDR profile), it may only allow the linear LMCS,and for another profile (i.e., HDR profile), it may only allow theregular LMCS, and for another profile, it may allow both regular LMCSand/or linear LMCS.

According to another embodiment of the present document, the LMCSpiecewise index identification process may be used in inverse lumamapping and chroma residual scaling. In this embodiment, theidentification process of the piecewise index may be used for blockswith chroma residual scaling enabled, and also invoked for all lumasamples in reshaped (mapped) domain. The present embodiment aims to keepits complexity low.

The following table shows the identification process (derivationprocess) of the existing piecewise function index.

TABLE 50 Identification of piecewise function index process for a lumasample  if ( lumaSample < LmcsPivot[ lmcs_min_bin_idx + 1 ] )   idxYInv= lmcs_min_bin_idx  else if ( lumaSample >= LmcsPivot[ LmcsMaxBinIdx ] )  idxYInv = LmcsMaxBinIdx  else {                       (8-1003)   for(idxYInv = lmcs_min_bin_idx; idxYInv < LmcsMaxBinIdx;   idxYInv++ ) {   if( lumaSample < LmcsPivot [ idxYInv + 1 ] )     break   }  }

In an example, in the piecewise index identification process, inputsamples may be classified into at least two categories. For example, theinput samples may be classified into three categories, first, second,and third categories. For example, the first category may representsamples (values thereof) less than LmcsPivot[lmcs_min_bin_idx+1], andthe second category may represent samples (values thereof) greater thanor equal to (values thereof) LmcsPivot[LmcsMaxBinIdx], The thirdcategory may indicate (values of) samples betweenLmcsPivot[lmcs_min_bin_idx+1] and LmcsPivot[LmcsMaxBinIdx].

In this embodiment, it is proposed to optimize the identificationprocess by eliminating the categories classifications. This is becausethe input to the piecewise index identification process are luma valuesin reshaped (mapped) domain, there should be no values beyond the mappedvalues at the pivot points lmcs_min_bin_idx and LmcsMaxBinIdx+1.Accordingly, the conditional process to classify samples into categoriesin the existing piecewise index identification process is unnecesary.For more details, specific examples will be described below with tables.

In an example according to this embodiment, the identification processincluded in Table 50 may be replaced with one of Tables 51 or 52 below.Referring to Tables 51 and 52, the first two categories of Table 50 maybe removed, and for the last category, the boundary value (the secondboundary value or the ending point) in the iterative for loop is changedfrom LmcsMaxBinIdx to LmcsMaxBinIdx+1. That is, the identificationprocess may be simplified and the complexity for the piecewise indexderivation may be reduced. Accordingly, LMCS-related coding can beefficiently performed according to the present embodiment.

TABLE 51 for( idxYInv = lmcs_min_bin_idx; idxYInv <= LmcsMaxBinIdx;idxYInv++ ) {  if( lumaSample < LmcsPivot [ idxYInv + 1 ] )   break }

TABLE 52 for( idxYInv = lmcs_min_bin_idx; idxYInv < LmcsMaxBinIdx+1;idxYInv++ ) {  if( lumaSample < LmcsPivot [ idxYInv + 1 ] )   break }

Referring to Table 52, comparison process corresponding to the conditionof the if statement (an equation corresponding to the condition of theif statement) may be iteratively performed on all of the bin indicesfrom the minimum bin index to the maximum bin index. The bin index, inthe case where the equation corresponding to the condition of the ifstatement is true, may be derived as the inverse mapping index forinverse luma mapping (or the inverse scaling index for chroma residualscaling). Based on the inverse mapping index, modified reconstructedluma samples (or scaled chroma residual samples) may be derived.

In the existing embodiment, for slices (e.g., intra slices) coded intoseparate block trees, the latency due to the chroma residual scalingdependence in the corresponding luma block was higher than the slicescoded into the dual tree. For this reason, in the existing embodiment,the LMCS chroma residual scaling was not applied to slices coded intoseparate block trees.

In this embodiment, chroma residual scaling may be applied even toslices coded into separate trees. When the single chroma residualscaling factor described above is used, there may be no latencyresulting from the application of the chroma residual scaling becausethere is no dependence between chroma residual scaling in acorresponding luma block.

The following tables show syntax and semantics of a slice headeraccording to the present embodiment.

TABLE 53 slice_header( ) { Descriptor  ...  if( sps_lmcs_enabled_flag ){   slice_lmcs_enabled_flag u(1)   if( slice_lmcs_enabled_flag ) {   slice_lmcs_aps_id u(5)    slice_chroma_residual_scale_flag u(1)   } ... }

TABLE 54 slice_chroma_residual_scale_flag equal to 1 specifies thatchroma residual scaling is enabled for the current slice, slice_chroma_residual_ scale_flag equal to 0 specifies that chroma residualscaling is not enabled for the current slice. When slice_chroma_residual_scale_flag is not present, it is inferred to be equal to0.

Referring to Table 54, regardless of a conditional clause indicatingwhether the current block has a dual tree structure or a single treestructure (or a flag for it), a chroma residual scaling flag may besignaled.

In an embodiment according to this document, ALF data and/or LMCS datamay be signaled in the APS. For example, 32 APSs may be used. In oneexample, if all APSs are used for ALF and/or LMCS, approximately 10Kbytes on-chip memory may be required for the buffer of the APSs. Inorder to limit (reduce) the memory required to store ALF/LMCSparameters, and the complexity of computation required for the LMCS,this embodiment proposes a method of limiting the number of ALFs and/orLMCS APSs.

In an example according to this embodiment, one LMCS model per picturemay be used (allowed) regardless of the number of slices or bricksincluded in the picture. The number of the APSs for the LMCS may be lessthan 32. For example, the number of the APSs for the LMCS may be 4.

The following table shows semantics related to the APS according to thepresent embodiment.

TABLE 55 adaptation_parameter_set_id provides an identifier for the APSfor reference by other syntax elements.  NOTE - APSs can be sharedacross pictures and can be different in  different slices within apicture.  For LMCS APS, different slices within a picture shall have thesame  LMCS APS.  The maximum number of LMCS APS shall be 4.aps_params_type specifies the type of APS parameters carried in the APSas specified in Table 3.2. Table 3.2 - APS parameters type codes andtypes of APS parameters aps_params_ Name of Type of APS typeaps_params_type parameters 0 ALF_APS ALF parameters 1 LMCS_APS LMCSparameters 2 . . . 7 Reserved Reserved

Referring to Table 55, the maximum number of LMCS APS may be determinedin advance. For example, the maximum number of LMCS APSs may be four.Referring to Tables 15 and 55, the plurality of APSs may include theLMCS APSs. The (maximum) number of LMCS APSs may be four. In oneexample, the LMCS data field included in one LMCS APS of the LMCS APSsmay be used in the LMCS procedure for the current block in the currentpicture.

The following table shows the semantics of the syntax elements includedin the slice header (or picture header).

TABLE 56 slice_lmcs_aps_id specifies the adaptation_parameter_set_id ofthe LMCS APS that the slice refers to. The TemporalId of the LMCS APSNAL unit having adaptation_parameter_set_id equal to slice_lmcs_aps_idshall be less than or equal to the TemporalId of the coded slice NALunit. At most one LMCS APS with the same value of adaptation_parameter_set_id may be referred to by two or more slices of the same picture.

The syntax element described from Table 56 may be described withreference to Table 54. In an example, the syntax elementslice_lmcs_aps_id may be included in the slice header. In anotherexample, the syntax element slice_lmcs_aps_id of Table 56 may beincluded in the picture header, and in this case, slice_lmcs_aps_id maybe modified to ph_lmcs_aps_id.

The following drawings are created to explain specific examples of thepresent specification. Since the names of specific devices described inthe drawings or the names of specific signals/messages/fields arepresented by way of example, the technical features of the presentspecification are not limited to the specific names used in thefollowing drawings.

FIGS. 16 and 17 schematically represent an example of a video/imageencoding method and associated components according to the embodiment(s)of this document. The method disclosed in FIG. 16 may be performed bythe encoding apparatus disclosed in FIG. 2 . Specifically, for example,S1600 of FIG. 16 may be performed by the predictor 220 of the encodingapparatus; S1610 may be performed by the residual processor 230, thepredictor 220, and/or the adder 250 of the encoding apparatus; S1620 maybe performed by the residual processor 230 or the predictor 220 of theencoding apparatus; S1630, S1640 and/or S1650 may be performed by theresidual processor 230 of the encoding apparatus; and S1660 may beperformed by the entropy encoder 240 of the encoding apparatus. Themethod disclosed in FIG. 16 may include the embodiments described abovein this document.

Referring to FIG. 16 , the encoding apparatus may generate predictionluma samples (S1600). With respect to the prediction luma samples, theencoding apparatus may derive the prediction luma samples of the currentblock based on the prediction mode. In this case, various predictionmethods disclosed in this document, such as inter prediction or intraprediction, may be applied.

The encoding apparatus may derive prediction chroma samples. Theencoding apparatus may derive the residual chroma samples based on theoriginal chroma samples and the prediction chroma samples of the currentblock. For example, the encoding apparatus may derive residual chromasamples based on the difference between the prediction chroma samplesand the original chroma samples.

The encoding apparatus may derive LMCS codewords (S1610). The encodingapparatus may derive respective LMCS codewords for a plurality of bins.For example, the above-described lmcsCW[i] may correspond to LMCScodewords derived by the encoding apparatus.

The encoding apparatus may generate mapped prediction luma samples(S1620). The encoding apparatus may generate mapped prediction lumasamples based on the LMCS codewords. For example, the encoding apparatusmay derive input values and mapping values (output values) of pivotpoints for luma mapping, and may generate mapped prediction luma samplesbased on the input values and mapping values. Here, the input values andthe mapping values may be derived based on the LMCS codewords. In anexample, the encoding device may derive a mapping index (idxY) based onthe first predicted luma sample, and may generate the first mappedpredicted luma sample based on the mapping value and the input value ofthe pivot point corresponding to the mapping index. In another example,linear mapping (linear reshaping, linear LMCS) may be used, and mappedprediction luma samples may be generated based on the forward mappingspooling factor derived from two pivot points in the linear mapping, andthus the index derivation procedure may be omitted due to the linearmapping.

The encoding apparatus may generate scaled residual chroma samples.Specifically, the encoding apparatus may derive a chroma residualscaling factor and generate scaled residual chroma samples based on thechroma residual scaling factor. Here, the chroma residual scaling of theencoding stage may be referred to as forward chroma residual scaling.Accordingly, the chroma residual scaling factor derived by the encodingapparatus may be referred to as a forward chroma residual scalingfactor, and forward scaled residual chroma samples may be generated.

The encoding apparatus may derive LMCS related information based on theLMCS codewords (S1630). Alternatively, the encoding apparatus maygenerate the LMCS related information based on the mapped predictionluma samples and/or the scaled residual chroma samples. The encodingapparatus may generate information on LMCS related information for thereconstructed samples. The encoding apparatus may derive an LMCS relatedparameter that may be applied for filtering the reconstructed samples,and may generate information on the LMCS related information based onthe LMCS related parameters. For example, the information on the LMCSrelated information may include information on the above-described lumamapping (e.g., forward mapping, inverse mapping, linear mapping),information on chroma residual scaling, and/or LMCS (or reshaping,reshaper) related indexes (e.g., a maximum bin index and a minimum binindex).

The encoding apparatus may generate residual luma samples based on themapped prediction luma samples (S1640). For example, the encodingapparatus may derive residual luma samples based on a difference betweenthe mapped prediction luma samples and original luma samples.

The encoding apparatus may derive residual information (S1650). Theencoding apparatus may derive residual information based on the scaledresidual chroma samples and/or the residual luma samples. The encodingapparatus may derive transform coefficients based on the transformprocedure for the scaled residual chroma samples and the luma residualsamples. For example, the transform process may include at least one ofDCT, DST, GBT, or CNT. The encoding apparatus may derive quantizedtransform coefficients based on the quantization process for thetransform coefficients. The quantized transform coefficients may have aone-dimensional vector form based on the coefficient scan order. Theencoding apparatus may generate residual information specifying thequantized transform coefficients. The residual information may begenerated through various encoding methods such as exponential Golomb,CAVLC, CABAC, and the like.

The encoding apparatus may encode the image/video information (S1660).The image information may include LMCS related information and/orresidual information. For example, the LMCS related information mayinclude information on the linear LMCS. In one example, at least oneLMCS codeword may be derived based on information on the linear LMCS.The encoded video/image information may be output in the form of abitstream. The bitstream may be transmitted to the decoding devicethrough a network or a storage medium.

The image/video information may include various information according toan embodiment of the present document. For example, the image/videoinformation may include information disclosed in at least one of Tables1 to 56 described above.

In an embodiment, the image information may include APSs. Each of theAPSs may include type information (e.g., aps_params_type) indicatingwhether or not the APS is an LMCS APS. An LMCS data field included inthe APS (e.g., the first LMCS APS) whose type information indicates thatthe APS is the LMCS APS may include the LMCS related information. TheLMCS data field may include information on the LMCS codewords. Themaximum number of LMCS APSs of the APSs may be a predetermined value.For example, the maximum number of LMCS APSs (the predetermined value)may be four.

In an embodiment, based on the value of the type information being 1,the APS may incorporate the LMCS data field including LMCS parameters.

In an embodiment, the image information may include header information.The header information may include LMCS related APS ID information. TheLMCS related APS ID information may represent an ID of an LMCS APS forthe current picture or the current block. For example, the headerinformation may be a picture header (or a slice header).

In an embodiment, the image information may include SPS. The SPS mayinclude a first LMCS enabled flag indicating whether or not the LMCSdata field is enabled.

In an embodiment, based on the value of the first LMCS enabled flagbeing 1, the header information may incorporate a second LMCS enabledflag indicating whether or not the LMCS data field in a picture isenabled.

In an embodiment, based on the value of the second LMCS enabled flagbeing 1, the LMCS related APS ID information may be incorporated intothe header information.

In one embodiment, when the current block has a single tree structure ora dual tree structure (when the current block has a separate treestructure, when the current block is coded into a separate tree), achroma residual scaling enabled flag indicating whether or not chromaresidual scaling is applied for the current block may be generated bythe encoding apparatus. When chroma residual scaling is applied to thecurrent picture, the current slice, and/or the current block, the valueof the chroma residual scaling enabled flag may be 1.

In an embodiment, a minimum bin index (e.g., lmcs_min_bin_idx) and/or amaximum bin index (e.g., LmcsMaxBinIdx) may be derived based on the LMCSrelated information. A first mapping value (LmcsPivot[lmcs_min_bin_idx])may be derived based on the minimum bin index. A second mapping value(LmcsPivot[LmcsMaxBinIdx] or LmcsPivot[LmcsMaxBinIdx+1]) may be derivedbased on the maximum bin index. Values of the reconstructed luma samples(e.g., lumaSample of Table 51 or 52) may be in a range from a firstmapping value to a second mapping value. In one example, values of allreconstructed luma samples may be in a range from a first mapping valueto a second mapping value. In another example, values of some samples ofthe reconstructed luma samples may be in a range from a first mappingvalue to a second mapping value.

In an embodiment, the image information may include a sequence parameterset (SPS). The SPS may include a linear LMCS enabled flag indicatingwhether or not the linear LMCS is enabled.

In an embodiment, the encoding apparatus may generate a piecewise indexfor chroma residual scaling. The encoding apparatus may derive a chromaresidual scaling factor based on the piecewise index. The encodingapparatus may generate scaled residual chroma samples based on theresidual chroma samples and the chroma residual scaling factor.

In an embodiment, the chroma residual scaling factor may be a singlechroma residual scaling factor.

In an embodiment, the LMCS related information may include informationon the linear LMCS and the LMCS data field. The information on thelinear LMCS may be referred to as information on linear mapping. TheLMCS data field may include a linear LMCS flag indicating whether or notthe linear LMCS is applied. When the value of the linear LMCS flag is 1,the mapped prediction luma samples may be generated based on informationon the linear LMCS.

In an embodiment, the information on the linear LMCS may includeinformation on a first pivot point (e.g., P1 in FIG. 12 ) andinformation on a second pivot point (e.g., P2 in FIG. 12 ). For example,the input value and mapping value of the first pivot point may be aminimum input value and a minimum mapping value, respectively. The inputvalue and mapping value of the second pivot point may be a maximum inputvalue and a maximum mapping value, respectively. An input value betweenthe minimum input value and the maximum input value may be linearlymapped.

In one embodiment, the image information includes information on themaximum input value and information on the maximum mapped value. Themaximum input value is equal to a value of the information on themaximum input value (i.e., lmcs_max_input in Table 33). The maximummapped value is equal to a value of the information on the maximummapped value (i.e., lmcs_max_mapped in Table 33).

In one embodiment, the information on the linear mapping includesinformation on an input delta value of the second pivot point (i.e.,lmcs_max_input_delta in Table 35) and information on a mapped deltavalue of the second pivot point (i.e., lmcs_max_mapped_delta in Table35). The maximum input value may be derived based on the input deltavalue of the second pivot point, and the maximum mapped value may bederived based on the mapped delta value of the second pivot point.

In one embodiment, the maximum input value and the maximum mapped valuemay be derived based on at least one equation included in Table 36described above.

In one embodiment, generating the mapped prediction luma samplescomprises deriving a forward mapping scaling factor (i.e.,ScaleCoeffSingle) for the prediction luma samples, and generating themapped prediction luma samples based on the forward mapping scalingfactor. The forward mapping scaling factor may be a single factor forthe prediction luma samples.

In one embodiment, the forward mapping scaling factor may be derivedbased on at least one equation included in Tables 36 and/or 38 describedabove.

In one embodiment, the mapped prediction luma samples may be derivedbased on at least one equation included in Table 38 described above.

In one embodiment, the encoding apparatus may derive an inverse mappingscaling factor (i.e., InvScaleCoeffSingle) for the reconstructed lumasamples (i.e., lumaSample). Also, the encoding apparatus may generatemodified reconstructed luma samples (i.e., invSample) based on thereconstructed luma samples and the inverse mapping scaling factor. Theinverse mapping scaling factor may be a single factor for thereconstructed luma samples.

In one embodiment, the inverse mapping scaling factor may be derivedusing a piecewise index derived based on the reconstructed luma samples.

In one embodiment, the piecewise index may be derived based on Table 51described above. That is, the comparison process included in Table 51(lumaSample<LmcsPivot[idxYInv+1]) may be iteratively performed from thepiecewise index being the minimum bin index to the piecewise index beingthe maximum bin index.

In one embodiment, the inverse mapping scaling factor may be derivedbased on at least one of equations included in Tables 33, 34, 35, and36, or Equation 11 or 12 described above.

In one embodiment, the modified reconstructed luma samples may bederived based on Equation 20, Equation 21, Table 39, and/or Table 40described above.

In one embodiment, the LMCS related information may include informationon the number of bins for deriving the mapped prediction luma samples(i.e., lmcs_num_bins_minus1 in Table 41). For example, the number ofpivot points for luma mapping may be set equal to the number of bins. Inone example, the encoding apparatus may generate the delta input valuesand delta mapped values of the pivot points by the number of bins,respectively. In one example, the input values and mapped values of thepivot points are derived based on the delta input values (i.e.,lmcs_delta_input_cw[i] in Table 41) and the delta mapped values (i.e.,lmcs_delta_mapped_cw[i] in Table 41), and the mapped prediction lumasamples may be generated based on the input values (i.e.,LmcsPivot_input[i] of Table 42) and the mapped values (i.e.,LmcsPivot_mapped[i] of Table 42).

In one embodiment, the encoding apparatus may derive an LMCS deltacodeword based on at least one LMCS codeword and an original codeword(OrgCW) included in the LMCS related information, and mapped lumaprediction samples may be derived based on the at least one LMCScodeword and the original codeword. In one example, the information onthe linear mapping may include information on the LMCS delta codeword.

In one embodiment, the at least one LMCS codeword may be derived basedon the summation of the LMCS delta codeword and the OrgCW, for example,OrgCW is (1<<BitDepthY)/16, where BitDepthY represents a luma bit depth.This embodiment may be based on Equation 14.

In one embodiment, the at least one LMCS codeword may be derived basedon the summation of the LMCS delta codeword andOrgCW*(lmcs_max_bin_idx−lmcs_min_bin_idx+1), for example,lmcs_max_bin_idx and lmcs_min_bin_idx are a maximum bin index and aminimum bin index, respectively, and OrgCW may be (1<<BitDepthY)/16.This embodiment may be based on Equations 15 and 16.

In one embodiment, the at least one LMCS codeword may be a multiple oftwo.

In one embodiment, when the luma bit depth (BitDepthY) of thereconstructed luma samples is higher than 10, the at least one LMCScodeword may be a multiple of 1<<(BitDepthY−10).

In one embodiment, the at least one LMCS codeword may be in the rangefrom (OrgCW>>1) to (OrgCW<<1)−1.

FIGS. 18 and 19 schematically represent an example of an image/videodecoding method and associated components according to the embodiment ofthis document. The method disclosed in FIG. 18 may be performed by thedecoding apparatus disclosed in FIG. 3 . Specifically, for example,S1800 of FIG. 18 may be performed by the entropy decoder 310 of thedecoding apparatus; S1810 may be performed by the predictor 330 of thedecoding apparatus; S1820 may be performed by the residual processor320, the predictor 330 and/or the adder 340 of the decoding apparatus;and S1830 may be performed by the adder 340 of the decoding apparatus.The method disclosed in FIG. 18 may include the embodiments describedabove in this document.

Referring to FIG. 18 , the decoding apparatus may receive/obtainvideo/image information (S1800). The video/image information may includeLMCS related information and/or residual information. For example, theLMCS related information may include information on the luma mapping(i.e., forward mapping, inverse mapping, linear mapping), information onchroma residual scaling, and/or indices (i.e., a maximum bin index, aminimum bin index, mapping index) related to LMCS (or reshaping,reshaper). The decoding aratus may receive/obtain the image/videoinformation through a bitstream.

The image/video information may include various information according toan embodiment of the present document. For example, the image/videoinformation may include information disclosed in at least one of Tables1 to 56 described above.

The decoding apparatus may generate prediction luma samples (S1810). Thedecoding apparatus may derive prediction luma samples of the currentblock in the current picture based on the prediction mode. In this case,various prediction methods disclosed in this document, such as interprediction or intra prediction, may be applied.

The image information may include residual information. The decodingapparatus may generate residual chroma samples based on the residualinformation. Specifically, the decoding apparatus may derive quantizedtransform coefficients based on the residual information. The quantizedtransform coefficients may have a one-dimensional vector form based on acoefficient scan order. The decoding apparatus may derive transformcoefficients based on a dequantization procedure for the quantizedtransform coefficients. The decoding apparatus may derive residualchroma samples and/or residual luma samples based on the transformcoefficients.

The decoding apparatus may derive LMCS codewords (S1820). The decodingapparatus may derive LMCS codewords based on LMCS related information.The decoding apparatus may derive information on a plurality of bins andinformation on LMCS codewords based on the LMCS related information. Thedecoding apparatus may derive respective LMCS codewords for a pluralityof bins. For example, the above-described lmcsCW[i] may correspond tothe LMCS codewords derived based on the LMCS related information.

The decoding apparatus may generate mapped prediction luma samples(S1830). The decoding apparatus may generate the mapped prediction lumasamples based on the prediction luma samples and the LMCS codewords. Forexample, the decoding apparatus may derive input values and mappedvalues (output values) of pivot points for luma mapping, and maygenerate mapped prediction luma samples based on the input values andmapped values. In one example, the decoding apparatus may derive a(forward) mapping index (idxY) based on the first prediction lumasample, and the first mapped prediction luma sample may be generatedbased on the input value and the mapped value of the pivot pointcorresponding to the mapping index. In other example, linear mapping(linear reshaping, linear LMCS) may be used and mapped prediction lumasamples may be generated based on a forward mapping scaling factorderived from two pivot points in the linear mapping, thus the indexderivation process may be omitted due to linear mapping.

The decoding apparatus may generate reconstructed luma samples (S1840).The decoding apparatus may generate reconstructed luma samples for thecurrent block based on the mapped prediction luma samples. Specifically,the decoding apparatus may sum the residual luma samples with the mappedprediction luma samples, and may generate reconstructed luma samplesbased on the result of the summation.

The decoding apparatus may generate scaled residual chroma samples.Specifically, the decoding apparatus may derive a chroma residualscaling factor and generate scaled residual chroma samples based on thechroma residual scaling factor. Here, the chroma residual scaling of thedecoding side may be referred to as an inverse chroma residual scaling,as opposed to the encoding side. Accordingly, the chroma residualscaling factor derived by the decoding apparatus may be referred to asan inverse chroma residual scaling factor, and inverse scaled residualchroma samples may be generated.

The decoding apparatus may generate reconstructed chroma samples. Thedecoding apparatus may generate reconstructed chroma samples based onthe scaled residual chroma samples. Specifically, the decoding apparatusmay perform the prediction process for the chroma component and maygenerate prediction chroma samples. The decoding apparatus may generatereconstructed chroma samples based on the summation of the predictionchroma samples and the scaled residual chroma samples.

In an embodiment, the image information may include APSs. Each of theAPSs may include type information indicating whether or not the APS isan LMCS APS. An LMCS data field included in the APS of which the typeinformation indicates that the APS is the LMCS APS may include the LMCSrelated information. The LMCS codewords may be derived based on the LMCSdata field. The maximum number of LMCS APSs of the APSs may be apredetermined value. For example, the maximum number of LMCS APSs (thepredetermined value) may be four.

In an embodiment, based on the value of the type information being 1,the APS may incorporate the LMCS data field including LMCS parameters.

In an embodiment, the image information may include header information.The header information may include LMCS related APS ID information. TheLMCS related APS ID information may represent an ID of an LMCS APS forthe current picture or the current block. For example, the headerinformation may be a picture header (or a slice header).

In an embodiment, the image information may include SPS. The SPS mayinclude a first LMCS enabled flag indicating whether or not the LMCSdata field is enabled.

In an embodiment, based on the value of the first LMCS enabled flagbeing 1, the header information may incorporate a second LMCS enabledflag indicating whether or not the LMCS data field in a picture isenabled.

In an embodiment, based on the value of the second LMCS enabled flagbeing 1, the LMCS related APS ID information may be incorporated intothe header information.

In one embodiment, when the current block has a single tree structure ora dual tree structure (when the current block has a separate treestructure, when the current block is coded into a separate tree), achroma residual scaling enabled flag indicating whether or not chromaresidual scaling is applied for the current block may be signaled. Whenchroma residual scaling is applied to the current picture, the currentslice, and/or the current block, the value of the chroma residualscaling enabled flag may be 1.

In an embodiment, a minimum bin index (e.g., lmcs_min_bin_idx) and/or amaximum bin index (e.g., LmcsMaxBinIdx) may be derived based on the LMCSrelated information. A first mapping value (LmcsPivot[lmcs_min_bin_idx])may be derived based on the minimum bin index. A second mapping value(LmcsPivot[LmcsMaxBinIdx] or LmcsPivot[LmcsMaxBinIdx+1]) may be derivedbased on the maximum bin index. Values of the reconstructed luma samples(e.g., lumaSample of Table 51 or 52) may be in a range from a firstmapping value to a second mapping value. In one example, values of allreconstructed luma samples may be in a range from a first mapping valueto a second mapping value. In another example, values of some samples ofthe reconstructed luma samples may be in a range from a first mappingvalue to a second mapping value.

In an embodiment, the image information may include a sequence parameterset (SPS). The SPS may include a linear LMCS enabled flag indicatingwhether or not the linear LMCS is enabled.

In an embodiment, a piecewise index (e.g., idxYInv in Tables 35, 36, or37) may be identified based on the information on the LMCS relatedinformation. The decoding apparatus may derive a chroma residual scalingfactor based on the piecewise index. The decoding apparatus may generatescaled residual chroma samples based on the residual chroma samples andthe chroma residual scaling factor.

In an embodiment, the chroma residual scaling factor may be a singlechroma residual scaling factor.

In an embodiment, the LMCS related information may include informationon the linear LMCS and the LMCS data field. The information on thelinear LMCS may be referred to as information on linear mapping. TheLMCS data field may include a linear LMCS flag indicating whether or notthe linear LMCS is applied. When the value of the linear LMCS flag is 1,the mapped prediction luma samples may be generated based on informationon the linear LMCS.

In an embodiment, the information on the linear LMCS may includeinformation on a first pivot point (e.g., P1 in FIG. 12 ) andinformation on a second pivot point (e.g., P2 in FIG. 12 ). For example,the input value and mapping value of the first pivot point may be aminimum input value and a minimum mapping value, respectively. The inputvalue and mapping value of the second pivot point may be a maximum inputvalue and a maximum mapping value, respectively. An input value betweenthe minimum input value and the maximum input value may be linearlymapped.

In an embodiment, the image information may include information aboutthe maximum input value and information about the maximum mapping value.The maximum input value may be the same as a value of information aboutthe maximum input value (e.g., lmcs_max_input in Table 33). The maximummapping value may be the same as a value of information about themaximum mapping value (e.g., lmcs_max_mapped in Table 33).

In an embodiment, the information on the linear mapping may includeinformation on the input delta value of the second pivot point (e.g.,lmcs_max_input_delta of Table 35) and information on the mapping deltavalue of the second pivot point (e.g., lmcs_max_mapped_delta of Table35). The maximum input value may be derived based on the input deltavalue of the second pivot point, and the maximum mapping value may bederived based on the mapping delta value of the second pivot point.

In an embodiment, the maximum input value and the maximum mapping valuemay be derived based on at least one equation included in Table 36described above.

In one embodiment, generating the mapped prediction luma samplescomprises deriving a forward mapping scaling factor (i.e.,ScaleCoeffSingle) for the prediction luma samples, and generating themapped prediction luma samples based on the forward mapping scalingfactor. The forward mapping scaling factor may be a single factor forthe prediction luma samples.

In one embodiment, the inverse mapping scaling factor may be derivedusing a piecewise index derived based on the reconstructed luma samples.

In one embodiment, the piecewise index may be derived based on Table 51described above. That is, the comparison process included in Table 51(lumaSample<LmcsPivot[idxYInv+1]) may be iteratively performed from thepiecewise index being the minimum bin index to the piecewise index beingthe maximum bin index.

In one embodiment, the forward mapping scaling factor may be derivedbased on at least one equation included in Tables 36 and/or 38 describedabove.

In one embodiment, the mapped prediction luma samples may be derivedbased on at least one equation included in Table 38 described above.

In one embodiment, the decoding apparatus may derive an inverse mappingscaling factor (i.e., InvScaleCoeffSingle) for the reconstructed lumasamples (i.e., lumaSample). Also, the decoding apparatus may generatemodified reconstructed luma samples (i.e., invSample) based on thereconstructed luma samples and the inverse mapping scaling factor. Theinverse mapping scaling factor may be a single factor for thereconstructed luma samples.

In one embodiment, the inverse mapping scaling factor may be derivedbased on at least one of equations included in Tables 33, 34, 35, and36, or Equation 11 or 12 described above.

In one embodiment, the modified reconstructed luma samples may bederived based on Equation 20, Equation 21, Table 39, and/or Table 40described above.

In one embodiment, the LMCS related information may include informationon the number of bins for deriving the mapped prediction luma samples(i.e., lmcs_num_bins_minus1 in Table 41). For example, the number ofpivot points for luma mapping may be set equal to the number of bins. Inone example, the decoding apparatus may generate the delta input valuesand delta mapped values of the pivot points by the number of bins,respectively. In one example, the input values and mapped values of thepivot points are derived based on the delta input values (i.e.,lmcs_delta_input_cw[i] in Table 41) and the delta mapped values (i.e.,lmcs_delta_mapped_cw[i] in Table 41), and the mapped prediction lumasamples may be generated based on the input values (i.e.,LmcsPivot_input[i] of Table 42) and the mapped values (i.e.,LmcsPivot_mapped[i] of Table 42).

In one embodiment, the decoding apparatus may derive an LMCS deltacodeword based on at least one LMCS codeword and an original codeword(OrgCW) included in the LMCS related information, and mapped lumaprediction samples may be derived based on the at least one LMCScodeword and the original codeword. In one example, the information onthe linear mapping may include information on the LMCS delta codeword.

In one embodiment, the at least one LMCS codeword may be derived basedon the summation of the LMCS delta codeword and the OrgCW, for example,OrgCW is (1<<BitDepthY)/16, where BitDepthY represents a luma bit depth.This embodiment may be based on Equation 14.

In one embodiment, the at least one LMCS codeword may be derived basedon the summation of the LMCS delta codeword andOrgCW*(lmcs_max_bin_idx−lmcs_min_bin_idx+1), for example,lmcs_max_bin_idx and lmcs_min_bin_idx are a maximum bin index and aminimum bin index, respectively, and OrgCW may be (1<<BitDepthY)/16.This embodiment may be based on Equations 15 and 16.

In one embodiment, the at least one LMCS codeword may be a multiple oftwo.

In one embodiment, when the luma bit depth (BitDepthY) of thereconstructed luma samples is higher than 10, the at least one LMCScodeword may be a multiple of 1<<(BitDepthY−10).

In one embodiment, the at least one LMCS codeword may be in the rangefrom (OrgCW>>1) to (OrgCW<<1)−1.

In the above-described embodiment, the methods are described based onthe flowchart having a series of steps or blocks. The present disclosureis not limited to the order of the above steps or blocks. Some steps orblocks may occur simultaneously or in a different order from other stepsor blocks as described above. Further, those skilled in the art willunderstand that the steps shown in the above flowchart are notexclusive, that further steps may be included, or that one or more stepsin the flowchart may be deleted without affecting the scope of thepresent disclosure.

The method according to the above-described embodiments of the presentdocument may be implemented in software form, and the encoding deviceand/or decoding device according to the present document is, forexample, may be included in the device that performs the imageprocessing of a TV, a computer, a smart phone, a set-top box, a displaydevice, etc.

When the embodiments in the present document are implemented insoftware, the above-described method may be implemented as a module(process, function, etc.) that performs the above-described function. Amodule may be stored in a memory and executed by a processor. The memorymay be internal or external to the processor, and may be coupled to theprocessor by various well-known means. The processor may include anapplication-specific integrated circuit (ASIC), other chipsets, logiccircuits, and/or data processing devices. Memory may include read-onlymemory (ROM), random access memory (RAM), flash memory, memory cards,storage media, and/or other storage devices. That is, the embodimentsdescribed in the present document may be implemented and performed on aprocessor, a microprocessor, a controller, or a chip. For example, thefunctional units shown in each figure may be implemented and performedon a computer, a processor, a microprocessor, a controller, or a chip.In this case, information on instructions or an algorithm forimplementation may be stored in a digital storage medium.

In addition, the decoding apparatus and the encoding apparatus to whichthe present disclosure is applied may be included in a multimediabroadcasting transmission/reception apparatus, a mobile communicationterminal, a home cinema video apparatus, a digital cinema videoapparatus, a surveillance camera, a video chatting apparatus, areal-time communication apparatus such as video communication, a mobilestreaming apparatus, a storage medium, a camcorder, a VoD serviceproviding apparatus, an Over the top (OTT) video apparatus, an Internetstreaming service providing apparatus, a three-dimensional (3D) videoapparatus, a teleconference video apparatus, a transportation userequipment (i.e., vehicle user equipment, an airplane user equipment, aship user equipment, etc.) and a medical video apparatus and may be usedto process video signals and data signals. For example, the Over the top(OTT) video apparatus may include a game console, a blue-ray player, aninternet access TV, a home theater system, a smart phone, a tablet PC, aDigital Video Recorder (DVR), and the like.

Furthermore, the processing method to which the present document isapplied may be produced in the form of a program that is to be executedby a computer and may be stored in a computer-readable recording medium.Multimedia data having a data structure according to the presentdisclosure may also be stored in computer-readable recording media. Thecomputer-readable recording media include all types of storage devicesin which data readable by a computer system is stored. Thecomputer-readable recording media may include a BD, a Universal SerialBus (USB), ROM, PROM, EPROM, EEPROM, RAM, CD-ROM, a magnetic tape, afloppy disk, and an optical data storage device, for example.Furthermore, the computer-readable recording media includes mediaimplemented in the form of carrier waves (i.e., transmission through theInternet). In addition, a bitstream generated by the encoding method maybe stored in a computer-readable recording medium or may be transmittedover wired/wireless communication networks.

In addition, the embodiments of the present document may be implementedwith a computer program product according to program codes, and theprogram codes may be performed in a computer by the embodiments of thepresent document. The program codes may be stored on a carrier which isreadable by a computer.

FIG. 20 shows an example of a content streaming system to whichembodiments disclosed in the present document may be applied.

Referring to FIG. 20 , the content streaming system to which theembodiment(s) of the present document is applied may largely include anencoding server, a streaming server, a web server, a media storage, auser device, and a multimedia input device.

The encoding server compresses content input from multimedia inputdevices such as a smartphone, a camera, a camcorder, etc. Into digitaldata to generate a bitstream and transmit the bitstream to the streamingserver. As another example, when the multimedia input devices such assmartphones, cameras, camcorders, etc. directly generate a bitstream,the encoding server may be omitted.

The bitstream may be generated by an encoding method or a bitstreamgenerating method to which the embodiment(s) of the present disclosureis applied, and the streaming server may temporarily store the bitstreamin the process of transmitting or receiving the bitstream.

The streaming server transmits the multimedia data to the user devicebased on a user's request through the web server, and the web serverserves as a medium for informing the user of a service. When the userrequests a desired service from the web server, the web server deliversit to a streaming server, and the streaming server transmits multimediadata to the user. In this case, the content streaming system may includea separate control server. In this case, the control server serves tocontrol a command/response between devices in the content streamingsystem.

The streaming server may receive content from a media storage and/or anencoding server. For example, when the content is received from theencoding server, the content may be received in real time. In this case,in order to provide a smooth streaming service, the streaming server maystore the bitstream for a predetermined time.

Examples of the user device may include a mobile phone, a smartphone, alaptop computer, a digital broadcasting terminal, a personal digitalassistant (PDA), a portable multimedia player (PMP), navigation, a slatePC, tablet PCs, ultrabooks, wearable devices (ex. Smartwatches, smartglasses, head mounted displays), digital TVs, desktops computer, digitalsignage, and the like. Each server in the content streaming system maybe operated as a distributed server, in which case data received fromeach server may be distributed.

Each server in the content streaming system may be operated as adistributed server, and in this case, data received from each server maybe distributed and processed.

The claims described herein may be combined in various ways. Forexample, the technical features of the method claims of the presentdocument may be combined and implemented as an apparatus, and thetechnical features of the apparatus claims of the present document maybe combined and implemented as a method. In addition, the technicalfeatures of the method claim of the present document and the technicalfeatures of the apparatus claim may be combined to be implemented as anapparatus, and the technical features of the method claim of the presentdocument and the technical features of the apparatus claim may becombined and implemented as a method.

What is claimed is:
 1. An image decoding method performed by a decodingapparatus, the method comprising: obtaining image information includingprediction related information and luma mapping with chroma scaling(LMCS) related information from a bitstream; generating prediction lumasamples for a current block in a current picture based on the predictionrelated information; deriving LMCS codewords based on the LMCS relatedinformation; generating mapped prediction luma samples based on theprediction luma samples and the LMCS codewords; and generatingreconstructed luma samples for the current block based on the mappedprediction luma samples, wherein: the image information includesadaptation parameter sets (APSs); an APS includes type informationindicating whether or not the APS is an LMCS APS; an LMCS data fieldincluded in the APS of which the type information indicates that the APSis the LMCS APS includes the LMCS related information; the LMCScodewords are derived based on the LMCS data field; a maximum number ofLMCS APSs of the APSs is a specific value; and the specific value isfour.
 2. The image decoding method of claim 1, wherein based on a valueof the type information being 1, the APS includes the LMCS data fieldincluding LMCS parameters.
 3. The image decoding method of claim 1,wherein: the image information includes header information; the headerinformation includes LMCS related APS ID information; and the LMCSrelated APS ID information represents an ID of an LMCS APS for thecurrent picture or the current block.
 4. The image decoding method ofclaim 3, wherein the image information includes a sequence parameter set(SPS), and wherein the SPS includes a first LMCS enabled flag indicatingwhether or not the LMCS data field is enabled.
 5. The image decodingmethod of claim 4, wherein based on a value of the first LMCS enabledflag being 1, the header information includes a second LMCS enabled flagindicating whether or not the LMCS data field in a picture is enabled.6. The image decoding method of claim 5, wherein based on a value of thesecond LMCS enabled flag being 1, the LMCS related APS ID information isincluded in the header information.
 7. The image decoding method ofclaim 1, wherein the image information includes a sequence parameter set(SPS), and wherein the SPS includes a linear LMCS enabled flagindicating whether or not a linear LMCS is enabled.
 8. The imagedecoding method of claim 1, wherein: the image information includesresidual information; residual chroma samples are generated based on theresidual information; a piecewise index is identified based on the LMCSrelated information; a chroma residual scaling factor is derived basedon the piecewise index; and scaled residual chroma samples are generatedbased on the residual chroma samples and the chroma residual scalingfactor.
 9. The image decoding method of claim 8, wherein based on thecurrent block having a single tree structure or a dual tree structure, achroma residual scaling enabled flag indicating whether or not chromaresidual scaling is applied for the current block is parsed.
 10. Theimage decoding method of claim 9, wherein the chroma residual scalingfactor is a single chroma residual scaling factor.
 11. The imagedecoding method of claim 1, wherein: the LMCS related informationincludes information on a linear LMCS; the LMCS data field includes alinear LMCS flag indicating whether or not the linear LMCS is applied;and based on a value of the linear LMCS flag being 1, the mappedprediction luma samples are generated based on the information on thelinear LMCS.
 12. The image decoding method of claim 10, wherein: theinformation on the linear LMCS includes information on a first pivotpoint and information on a second pivot point; an input value and amapping value of the first pivot point are a minimum input value and aminimum mapping value, respectively; an input value and a mapping valueof the second pivot point are a maximum input value and a maximummapping value, respectively; and an input value between the minimuminput value and the maximum input value is linearly mapped.
 13. Theimage decoding method of claim 10, wherein: the image informationincludes information on maximum input value and information on maximummapping value; the maximum input value is equal to a value of theinformation on the maximum input value; and the maximum mapping value isequal to a value of the information on the maximum mapping value. 14.The image decoding method of claim 1, wherein the generating the mappedprediction luma samples comprises: deriving a forward mapping scalingfactor for the prediction luma samples; and generating the mappedprediction luma samples based on the forward mapping scaling factor, andwherein the forward mapping scaling factor is a single factor for theprediction luma samples.
 15. The image decoding method of claim 1,further comprising: deriving an inverse mapping scaling factor for thereconstructed luma samples; and generating modified reconstructed lumasamples based on the reconstructed luma samples and the inverse mappingscaling factor, wherein the inverse mapping scaling factor is a singlefactor for the reconstructed luma samples.
 16. The image decoding methodof claim 15, wherein the inverse mapping scaling factor is derived usinga piecewise index derived based on the reconstructed luma samples. 17.An image encoding method performed by an encoding apparatus, the methodcomprising: generating a predicted luma sample for a current block in acurrent picture; deriving luma mapping with chroma scaling (LMCS)codewords; deriving LMCS related information based on the LMCScodewords; generating mapped prediction luma samples based on the LMCScodewords; generating residual luma samples for the current block basedon the mapped prediction luma samples; deriving residual informationbased on the residual luma samples; and encoding image informationincluding the LMCS related information and the residual information,wherein: the image information includes adaptation parameter sets(APSs); an APS includes type information indicating whether or not theAPS is an LMCS APS; an LMCS data field included in the APS of which thetype information indicates that the APS is the LMCS APS includes theLMCS related information; the LMCS data field includes information onthe LMCS codewords; a maximum number of LMCS APSs of the APSs is aspecific value; and the specific value is four.
 18. A non-transitorycomputer-readable storage medium storing a bitstream generated by amethod, the method comprising: generating a predicted luma sample for acurrent block in a current picture; deriving luma mapping with chromascaling (LMCS) codewords; deriving LMCS related information based on theLMCS codewords; generating mapped prediction luma samples based on theLMCS codewords; generating residual luma samples for the current blockbased on the mapped prediction luma samples; deriving residualinformation based on the residual luma samples; and encoding imageinformation including the LMCS related information and the residualinformation to output the bitstream, wherein: the image informationincludes adaptation parameter sets (APSs); an APS includes typeinformation indicating whether or not the APS is an LMCS APS; an LMCSdata field included in the APS of which the type information indicatesthat the APS is the LMCS APS includes the LMCS related information; theLMCS data field includes information on the LMCS codewords; a maximumnumber of LMCS APSs of the APSs is a specific value; and the specificvalue is four.